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1. Introduction 


Students of initial teacher education need to be prepared for the classrooms of tomorrow. 
Teachers require the necessary skills and competences to teach in culturally diverse contexts, 
to collaborate across disciplines, and to use technologies in innovative ways; but the Digital 
Economy and Society Index reports that only 20-25% of students in Europe are taught by teachers 
who are confident using technology in the classroom, and only a small number of students 
of education are gaining intercultural experience by participating in international mobility 
programmes such as Erasmus+ (European Commission, 2015a). It is clear that institutions of 
initial teacher education need to find new ways to prepare their students for the challenges they 


will meet in the future. 


Virtual exchange, also commonly referred to as telecollaboration or Collaborative Online 
International Learning (COIL), is a broad term used to describe different methods of engaging 
students in online intercultural collaboration projects with partner classes within their 
programmes of study and under the guidance of teachers or trained facilitators (O’ Dowd & Lewis, 


2016). Virtual exchange is based on student-centred, international, and collaborative approaches 


to learning where knowledge and understanding are constructed through interaction and 
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negotiation with students from other cultures. This approach has been employed in university 
education for over 20 years (Cummins & Sayers, 1995; Warschauer, 1996) and has been used 
widely in subject areas such as foreign language education, business studies, and initial teacher 


education (O’Dowad, 2018). 


There are various different models of virtual exchange. However, in the context of initial 
teacher education, the telecollaborative model of virtual exchange is commonly carried out in 


the following way: 


teacher trainers from two or more different countries come into contact and 
develop a joint curriculum for an exchange which will be integrated into their 


respective courses; 
during the semester, students from the different classes work together 
online, discussing issues related to their curricula and collaborating to create 


educational materials and activities; 


the teacher trainers guide students in their online interactions and in their 


reflections on this experience; and 


at the end of the exchange, students receive academic credit for their 


work by submitting portfolios, essays, or making presentations about their 


collaboration with their international partners. 


This process is illustrated in Figure 1 below. 


Recent years have seen the steady growth in interest and awareness of virtual exchange as a 
tool in university education in Europe. The European Commission has financed several virtual 
exchange projects. For example, between 2011 and 2014, the Integrating Telecollaborative 
Networks in Higher Education (INTENT) project was financed to achieve greater awareness of 
virtual exchange in university education, while Evidence-Validated Online Learning through 
Virtual Exchange (EVOLVE) aims to promote virtual exchange across all subject areas in higher 
education. In 2017, the European Commission also published a large scale feasibility study on 
the potential of virtual exchange, and, following this, launched Erasmus+ Virtual Exchange, a 
flagship programme which aims to expand the reach and scope of the Erasmus+ programme via 


virtual exchange. 


“a 


Teacher trainers from two 


or more different 
countries come into 
contact and develop a 
joint curriculum for an 
exchange which will be 
integrated into their 
respective courses. 
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The teacher trainers guide 
students in their online 
interactions and in their 

reflections on this 
experience. 


At the end of the 
exchange, students 
receive academic credit for 
their work by submitting 
portfolios, essays, or 
making presentations 
Eleleltimaitim@aelitllelela ela) 
with their international 


activities. 


partners. 


Figure 1. Stages of a telecollaborative virtual exchange in initial teacher education 


While there are many research studies which report on the impact of one particular virtual 
exchange (see Guth & Helm, 2010, and O’Dowd & Lewis, 2016, for some recent examples), there 
have not been to date any large scale studies which have demonstrated the learning gains of a 
large cohort of students in different exchanges who have taken part in one particular model of 
virtual exchange. With this in mind, the Evaluating and Upscaling Telecollaborative Teacher 
Education (EVALUATE) project aimed to gauge the impact of virtual exchange on students 
involved in initial teacher education. Between 2017-18, the project consortium trained teacher 
trainers from 34 initial teacher training institutions and organised virtual exchanges which 
involved over 1,000 students. We then analysed the learning gains from these exchanges using 
qualitative and quantitative research methodologies. Following that, we worked with partners 
from the participating ministries of education to take initial steps of promoting and upscaling 


the use of virtual exchange in teacher training across Europe. 


EVALUATE involved: 


25 virtual exchange projects between classes of initial teacher education at 


34 teacher training institutions; 


1,018 students of initial teacher education taking part in the virtual exchanges 


and the data collection process; and 


quantitative and qualitative research instruments collecting data on students’ 


digital, intercultural, and linguistic competence development as well as on the 


learning experiences of both students and teacher trainers. 
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This report presents the findings of our study. However, not only do we provide here a large- 
scale study to demonstrate the impact of virtual exchange in an important field of university 
education, we also explore how policy makers and university management can take actions to 
make this methodology more widespread across university education and support students and 


teachers in its use. 


In Section 2 we look at the challenges facing initial teacher education as regards the development 
of future teachers’ foreign language, intercultural, and digital competences. Section 3 outlines 
the research methodology used in the EVALUATE European policy experiment. Then, in Section 4 
and Section 5, we present the empirical findings of the study before moving on, in Section 6, to 
look at three case studies which help to contextualise these findings. Finally, in Section 7, we 
bring together the different outcomes of the study and examine how virtual exchange can be 


promoted by educational decision makers at regional, national, and European levels. 
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2. Background to the EVALUATE study 


2.1. Justification for the study 


Institutions of initial teacher education around Europe are being called upon to develop educational 
programmes which will prepare the teachers of tomorrow for the diverse array of challenges which 
they will encounter in their classrooms. These challenges include promoting inclusive approaches 
to education for all learners, including those of migrant origins and those from disadvantaged 
socioeconomic backgrounds (European Commission, 2018). They also involve introducing a European 
dimension of teaching which includes knowledge of foreign languages and the principles of active 
citizenship education (Council of Europe, 2016). Finally, they require the teachers of tomorrow to be 
able to integrate online technologies into their classrooms in ways which will promote both student- 
centred, collaborative learning and the development of critical digital competences (European 


Parliament, 2018). The following sections look briefly at these different challenges. 
2.1.1. | Developing student teachers’ intercultural skills and a European perspective 


The common approach until now to developing the intercultural and foreign language skills of 
student teachers and to enhancing their understanding of the European dimension of education 
has been to rely on international student mobility programmes such as Erasmus+. However, in 
comparison to other subject areas, the level of participation of students of education in Erasmus+ 
mobility has been disappointingly low. The most recent data available (see Figure 2) shows that 
only 3.41% of students in academic mobility come from education, while only 2.6% of students 
engaged in international work placements come from this area of studies (European Commission, 


2015a). 


There are also various reasons to question whether student mobility programmes are alone 
sufficient to help students meet their intercultural and foreign language learning goals. The first 
reason is that the financial costs of engaging in student mobility programmes mean that it is not 
economically viable as a way to develop intercultural or global competences in a large majority 
of university students (Richardson, 2016). The European Commission (2013), which has invested 


greatly in promoting student mobility at university level, recognises this: 


“mobility will always be limited to a relatively small percentage of the student and staff 


population: higher education policies must increasingly focus on the integration of a global 
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dimension in the design and content of all curricula and teaching/learning processes [...] to 
ensure that the large majority of learners, the 80-90% who are not internationally mobile 
for either degree or credit mobility, are nonetheless able to acquire the international skills 


required in a globalised world” (p. 6). 
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Figure 2. Share of subject areas in mobility for studies and work placements 2013-14 
( Pp. 9) 


A second reason for using virtual exchange as a complementary approach to student mobility is 
based on growing evidence in the literature that physical mobility does not lead automatically 
to the development of intercultural competence or an enhanced transnational identity - which 
are the very goals of internationalisation policies and mobility programmes. Papatsiba (2005), 
for example, looked at the impact of Erasmus mobility on a cohort of French students in order 
to investigate the extent to which students’ experiences reflected the political and policy aims of 


the Erasmus mobility programme and she concluded that 


“acquiring a feeling of belonging in an enlarged Europe, enriching national identities with 
the desired European dimension remained a somewhat random result of experiential 
learning. This type of learning depends on situations, on encounters, as well as on the 


individual’s psychology” (p. 183). 
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There have been similar findings by Paige et al. (2009) who argue that the keys to successful 
physical mobility programmes are how the exchanges are structured and the type of learning 


experiences which they provide. 


2.1.2. Preparing student teachers to use digital technologies in innovative ways 


A further challenge for initial teacher education is how to prepare student teachers to use online 
technologies to promote student-centred and innovative approaches to learning in classrooms. 
Although student teachers may belong to a generation referred to as ‘digital natives’ (Prensky, 
2006), research by Kirschner and De Bruyckere (2017) and Margaryan, Littlejohn, and Vojt (2011) 
has questioned the belief that they are somehow intuitively capable of using digital technologies 
in collaborative ways in their learning and teaching practices. Valtonen et al. (2011) looked at the 
academic practices of student teachers in Finland and found that “the technological knowledge 
of student teachers is not what would be expected for representatives of the Net Generation” 
(p. 13). They also looked at the technological pedagogical knowledge of these learners and found 
that they used a very limited range of software in their teaching practices and that when they 
used social media, it was as a passive source of information transmission and not as a tool for 


actively creating content, interacting with others, or sharing resources (Valtonen et al., 2011). 


Further research suggests that although digital tools and resources are increasingly available to 
educators, they continue to be used in a very limited, traditional manner by most teachers. The 
European Commission (2015b) reports that online technologies are mainly used as aremedial tool 


and that innovative approaches to using online technologies are often limited to the pedagogical 


activities of a small minority of innovative practitioners. Similarly, the 2015 report of the 
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European Council and Commission on the implementation of the Education and Training 2020 
states that high-quality learning requires teachers to take a more active approach to innovative 
pedagogies and their application through digital tools and the report calls for teacher training 
programmes to “reap the benefits of new [Information and Communication Technology (ICT)] 
developments and adopt innovative and active pedagogies, based on participatory and project- 
based methods” (£12020, 2015, p. 5). The recent OECD (2015) study ‘Teaching with technology’ 
found that only 38% of secondary teachers reported frequently using online technologies in 


students’ projects or class work (Figure 3). 
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Figure 3. Teachers’ use of ICT for students’ projects or class work (© 


Against this background, the Education and Training 2020 Working Group on Schools Policy calls 
on European educational systems to move away from isolated classrooms towards new teaching 
methods which are based on collaboration. The working group proposes that teachers should 
be encouraged to incorporate both collaborative practices and a collaborative culture into their 
work and that training institutions should take steps to engage teachers in “networks, professional 
learning communities and other partnerships” (ET2020, 2015, p. 36). Similarly, the European 
Commission’s communication New Priorities for European Cooperation in Education and Training 


highlights the need to train educators to use ICT tools in combination with innovative pedagogies. 


In order for this to happen, it is clear that it is in the context of initial teacher education that 


future teachers should not only be trained in basic digital competences, but they should also be 
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exposed to and engaged in innovative applications of online technologies which involve their 
use in collaborative, student-centred approaches to learning. The European Parliament’s (2018) 


draft report on education in the digital era is clear about this: 


“The digital transformation does not only require education in digital skills. Rather, its 
implications also have the potential to transform teaching methods. Unfortunately, this 


potential is not being fully tapped into as teachers need to be educated themselves” (p. 8). 


2.1.3. | Moving forward with virtual exchange 


It is against this background of limited student mobility in teacher education and limited use 
of online technologies for innovative, collaborative approaches to learning that this study was 
undertaken. The study set out to explore the potential impact of an innovative and intercultural 
approach to online learning, virtual exchange, on students of initial teacher education and how 


it may contribute to the development of their intercultural, linguistic, and digital competences. 


The underlying argument for our study, as mentioned in (O’Dowd, 2017, p. 38), is that if the 
teachers of tomorrow are to engage their students in innovative and inclusive approaches to 
online learning, they first of all need to experience this type of learning themselves during 
their own study programmes. In European school education, virtual exchange has already 
been recognised as a powerful tool for the development of students’ competences in the form of 
eTwinning (Education for Change, 2013). However, it has been shown that one of the barriers to 
the success of eTwinning is that it is often not included in initial teacher education programmes. 
Whilst eTwinning offers continuous professional development for in-service teachers, we argue 
that the take-up of such virtual exchange initiatives will remain limited until student teachers 
are given the opportunity to experience these online learning experiences during their own 
training. EVALUATE took a first important step in integrating virtual exchange in a large number 
of initial teacher education institutions across Europe and undertook various training and 


promotional events to raise awareness of virtual exchange in teacher education. 


2.2. Research aims 


This research study was funded by the Erasmus+ KA3 programme (EACEA/34/2015) and was 
a European policy experiment. The aim of policy experimentations is to assess the relevance, 
effectiveness, and potential scalability of innovative policy measures through experimental 


or semi-experimental approaches. This particular European policy experiment evaluated the 
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impact of virtual exchange on student teachers involved in initial teacher education (also 


referred to as ‘pre-service’ education) in European countries and regions. 


The three key actors in European policy experimentations are the responsible public authorities, 
the researchers, and the target groups. In this particular case, the public authorities were 
from the regional governments in the European regions of Castilla y Leon in Spain and Baden 
Wurttemberg in Germany, and from the national ministries of education in Hungary, Spain, and 
Portugal. The researchers were a team of experienced researchers and practitioners in various 
European institutions, while the target groups were students studying in programmes of initial 
teacher education as well as their instructors (i.e. teacher trainers) who were undertaking to use 


virtual exchange in their classes. 


The guiding research question for the study was: 


Will participation in virtual exchange contribute to the development of 


competences which student teachers need to teach, collaborate, and innovate 


effectively in a digitalised and cosmopolitan world? 


The specific research questions were: 


What impact will virtual exchange have on student teachers’ digital- 


j Xero ketexoyen (or=V eo) aay oleic alesrg 


What impact will virtual exchange have on student teachers’ intercultural 


competence? 
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competence (this question was dealt with in contexts where foreign language 


learning was a part of the teacher education courses)? 


These questions were complemented by a fourth question which would also be of use to the 


public authorities in the participating regions and countries: 


What were the experiences of the teacher trainers who endeavoured to 


introduce virtual exchanges in their classrooms? 
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2.3. The EVALUATE virtual exchange programme 


Twenty five virtual exchanges were run as part of the EVALUATE study. A total of 17 exchanges 
ran in the winter semester 2017-18, while eight exchanges took place in the spring semester 
2018 (see Table 1 below for an overview). In total, 34 institutions of initial teacher education 
from 16 countries were involved (as some institutions participated in various exchanges). Most 
institutions were from European countries but teacher educators from the United States, Brazil, 
Israel, Turkey, Macau, and Canada also took part in the project and their institutions were 


included in the study. 


Altogether, a total of 1,018 students from 34 institutions were invited to take part in the 
EVALUATE study. All of these students were studying either undergraduate or Masters’ degrees 
in initial teacher education, but their subject areas varied. Some were studying subjects related 
to primary school education in general, while others were studying subjects related to foreign 
language education, bilingual education, mathematics, special needs, and physical education. 
Classes were matched depending on different variables including subject being studied, language 
choice, their courses’ start and end dates and the themes their teachers wished to focus on. The 
majority of classes involved students who were studying English as a foreign language. This 
is perhaps not surprising as virtual exchange is already quite well known in foreign language 
education and is seen as an effective tool for supporting communicative approaches to foreign 


language learning. 


Table 1. Overview of the EVALUATE virtual exchanges 


Poland & Germany Jan Dlugosz University, Poland & PH Heidelberg 


Portugal & Brazil mere Politécnico de Castelo Branco & Universidade Estadual 
aulista 

Spain & Holland Universidad de Burgos & University of Utrecht 

Spain & Poland Universidad de Valladolid & University of Warsaw 


Spain & Poland Maria Curie-Sklodowska University, Lublin & Universidad de Vic 


o Israel & Spain Salhi College for Teacher Education, Israel & Universidad de 
urgos 
Holland & Spain Leiden University & Universidad de Valladolid 
Poland & Spain Jagiellonian University & Universidad de Burgos 


Israel & Germany Kibbutzim College of Education, Technology and the Arts, Tel Aviv & 
Karlsruhe University of Education 
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r 
; 
: 
: 


14 | Sweden, Israel & Spain Malm6 University & Tel Aviv University & Universidad de Leon 
15 | Spain & Germany Universidad Complutense de Madrid & PH Freiburg 


16 | Portugal & Brazil Tncueale Politécnico de Castelo Branco & Universidade Federal de 
elotas 


Sweden & USA Malm University & The College at Brockport State University of New 
or. 
USA & Spain Texas A&M University & University of Burgos 


Portugal & Macau insur ute Politécnico de Castelo Branco & Instituto Politécnico de 
acau 
Spain & Poland University of Leon & University of Warsaw 


25 | Spain & USA Universitat Autonoma de Barcelona & University of Illinois at Urbana 
Champaign 


The treatment in our study involved engaging these classes of initial teacher education in a 


period of intensive virtual exchange with partner classes in institutions in other countries based 
on specifically-designed tasks and content related to digital-pedagogical competences as well as 
intercultural competence. Digital-pedagogical competence refers to the teacher’s ability to plan, 
implement, and evaluate the use of technologies (online tools and applications) in their classes 
(Koehler & Mishra, 2005). We understand intercultural competence as the set of skills, attitudes, 
and knowledge which learners need if they are to communicate effectively with members of 


other cultures. 


The virtual exchange model which was used in the exchanges was based on the Progressive 
Exchange Model which has been widely used in virtual exchange research and practice to date 
(O’Dowd & Lewis, 2016; O’Dowd & Ware, 2009). As mentioned in O’Dowd (2017), the model 
involves “three interrelated tasks which move from information exchange to comparing and 
analysing cultural practices and finally to working on a collaborative product” (p. 40) (see 
Figure 4). Teacher trainers participating in the study were provided with three sets of tasks so 
they could choose from various tasks at each stage of the task sequence. However, all the tasks 


available to the educators focussed on the development of the key competences and themes 
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which have been identified by the public authorities as key for teacher education in their 


countries or regions. 


¢ Task 1: Information Exchange: Getting to know each other 


© Student-Teachers learn how to use a range of online technologies to 
communicate and collaborate online 


¢ Task 2: Comparing and analyzing cultural practices 


¢ Student-Teachers develop their ability to communicate effectively in 
intercultural contexts and to compare cultural perspectives on 
educational issues such as bullying, multicultural classrooms etc. 


¢ Task 3: Working on a collaborative product 


¢ Student-Teachers develop their analytical and critical thinking skills 
and cooperation skills to collaborate with their international partners 
on the creation of an educational project for their teaching contexts 


Figure 4. The progressive exchange telecollaborative model for initial teacher 
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The recruitment and training of teacher trainers who were to take part in this study over the two 


rounds of field trials involved the following steps: 


dissemination of the EVALUATE project and recruitment of interested teachers 
through online networks such as Twitter, Facebook, and professional mailing 


lists as well as presentations of the project at international conferences; 


matching of teacher trainers with their partner-teachers based on criteria such 


as semester start and end dates, course objectives, and foreign language level; 


development of an online platform where the virtual exchanges could take 


place ( ); 


organisation of two teacher training workshops at the University of Padova, 


Italy (for exchanges running in Semester 1) and at the University of Leon, 


Spain (for exchanges running in Semester 2); and 
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assigning each virtual exchange partnership a mentor from the project 


consortium who provided support and expert advice to the teacher trainers 


as they ran their virtual exchanges. 


The timeline for this process is outlined in the Figure 5 and Figure 6 below. 


February-June 2017 May-June 2017 July 2017 July-September 2017 September 2017- 


7 ; fea ene anuary 2018 
Recruitment Development of a virtual Organisation of teacher Assigning a mentor J of 


of teacher trainers learning platform for training workshop for each virtual exchange 17 virtual exchanges 
the virtual exchanges in Padova and and pre-exchange take place 
partner-matching planning by teachers 


a (eLUla-momm CoLUL arom MoymYA(a4Ut-] m=> Colat-lave(-mil-] (emda tI hs 
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September-December 2017 February-June 2018 January 2018 January 2018 February-July 2018 


Recruitment of teacher Analysis of first round Organisation of teacher Assigning a mentor 8 virtual exchanges 
trainers for second round of exchanges training workshop in Léon for each virtual exchange take place 
of exchanges and partner-matching and pre-exchange 
planning by teachers 


a (eLUla-Momm CoLUL are mao) MYA adUr-] => Colat-lave(-mil-] (emda tI hs 


The workshops were offered free of charge to the participating teacher trainers and they were 
encouraged to use Erasmus Training funding from their universities to travel to the events. In 
total, 40 teacher trainers attended the Padova event, while 15 attended the Leén event. The 
Leon event was also followed online in its live webcast by seven teacher trainers. The lower 
attendance at the Leon workshop reflected the smaller number of exchanges (eight) which were 


planned to run in the second semester. 
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3. Research methodology of the EVALUATE study 


The process of evaluating virtual exchange in initial teacher education is complex and requires 
a comprehensive research methodology. The research team followed the J-Pal Europe (2016) 
recommendations in the Guidelines for Conducting a European Policy Experiment and used 
a mixed methods approach (Dornyei, 2007; Nunan & Bailey, 2009) using quantitative data to 
measure the development of students’ intercultural, digital-pedagogical competences, while 
qualitative data (in the form of reflective diaries and interviews) were used to answer questions 
related to why virtual exchange had the impact it did, how it was integrated in the participating 
institutions, and how the virtual exchange model impacted differently across various socio- 


institutional contexts. 

3.1. Quantitative research methods 
A pre- post-test design approach with a control group was implemented, in line with 
recommendations from Torgerson and Torgerson (2008) and Rogaten et al. (2018). In order to 
cross-validate the findings, two separate implementations (round 1: n=678; round 2, n=340) were 
conducted, which allowed us to test and (re-)confirm the initial findings. 


3.1.1. | Participants and settings 


In total 1,018 participants were included in the overall sample set, making this, to our knowledge, 


the largest research study of virtual exchange to have been carried out to date. Nine-hundred 
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and six participants completed the pre-test (see Section 3.1.2), leading to a response rate at 
the pre-test of 89%. Six-hundred and eighty-three (63%) participants completed the post-test, 
whereby 57% of participants completed both the pre- and post-test, which for longitudinal 
research in social science is a very good response rate. In total, there were 25 virtual exchanges 
with 17 exchanges being carried out in Round 1 and an additional eight exchanges in Round 
2. In total, 76 participants from three institutions were put into the control condition (i.e. no 
treatment), while the other students did receive a treatment. In total, 34 different organisations 
from 16 countries participated in the exchanges. In terms of the average duration of the virtual 


exchange, a typical exchange lasted for 67.09 days. 
3.1.2. Instruments used 


In order to establish the impact of virtual exchange on students’ intercultural competence, it was 
decided to build on the work of Chen and Starosta (1996) and a 20 item questionnaire developed 
and validated by Portalla and Chen (2010) was used to measure aspects of intercultural 
communicative competence. This tool looks at one overall construct of intercultural effectiveness 
and five main constructs. For each construct, we illustrate the number of items per construct 
in brackets, the respective Cronbach alpha, and one exemplary item: (1) interaction relaxation 
(five items, a=.80, “I find it is easy to talk with people from different cultures”); (2) behavioural 
flexibility (four items, a=.65, “I am afraid to express myself when interacting with people from 
different cultures (reverse coded)”); (3) interaction management (two items, a=.77, “I am able 
to express my ideas clearly when interacting with people from different cultures”); (4) message 
skills (three items, a=.66, “I have problems with grammar when interacting with people from 
different cultures (r)”); and (5) identity maintenance (three items, a=.58, “I find I have a lot in 
common with people from different cultures during our interactions”). Commonly, a Cronbach 
alpha >.60 indicates reasonable to good reliability. Pre- and post-test Cronbach alphas indicated 
reliable constructs, and pre-post intercultural communicative competence correlations indicated 


a rho=.705, p<.01. 


Building on the Technological Pedagogical Content Knowledge (TPACK) work of Koehler and 
Mishra (2005) (Figure 7), and as described in greater detail in Section 4, a 47 item questionnaire 
developed and validated by Schmidt et al. (2009) was also used. Given that participants had to 
fill in the intercultural competence and foreign language questionnaire as well during the pre- 
and post-test, we shortened the questionnaire to 17 items. The TPACK instrument consisted of 
one overall TPACK construct, and four main sub-constructs, whereby a Likert response scale of 


1 (=totally disagree) to 5 (=totally agree) was used. These four main constructs were: 
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Technology knowledge: knowledge about various technologies, ranging from 
low-tech technologies such as pencil and paper to digital technologies such as 


the Internet, digital video, interactive whiteboards, and software programmes. 


Technological pedagogical knowledge: knowledge of how various technologies 
(oreba olombtsicxeM bam ccre(olaveatcem-Dalomovatel-susje-baXeDbat:maar-umonsybarcmacrel aval) (ol-armunt-hvme atcualexe 


the way teachers teach. 


TPACK: knowledge required by teachers for integrating technology into their 
iccrcLooabbarembaur-vahymeo) acy amr Dust: Pad Kcr-Col acy ucm atchicr-DaMbalaovuuarcmovalelsvacie-talebbaleme) muaXs 
complex interplay between the three basic components of knowledge (content 
knowledge, pedagogical knowledge, technology knowledge) by teaching 


content using appropriate pedagogical methods and technologies. 
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new representations for specific content. [...] Teachers understand that, by 
using a specific technology, they can change the way learners practise and 
understand concepts in a specific content area ( : 


pp. 132-134). 


For each construct, we illustrate the number of items per construct in brackets, the respective 
Cronbach alpha, and one exemplary item: (1) technology knowledge (seven items, a=.85, “I can 
learn technology easily”); (2) technological pedagogical knowledge (four items, a=.73, “I can 
choose technologies that enhance the teaching approaches for a lesson”); (3) TPACK (five items, 
a=.81, “I can select technologies to use in my classroom that enhance what I teach, how I teach, 
and what students learn”); and (4) technological content knowledge (one item, “I can choose 
technologies that enhance the content for a lesson”). Pre- and post-test Cronbach alphas indicated 
reliable constructs, and pre-post TPACK correlations indicated a rho=.492, p<.01, indicating that 


we should treat this construct carefully as it does not meet normal conventions of >.60. 


An eight item questionnaire related to foreign language competence was also developed for 
the pre-test, and for the post-test a nine item questionnaire was used. One overall scale was 
constructed called pre-foreign language competence (eight items, a=.XX, “I want to ... be able 
to understand foreign language speakers better”) and post-foreign language competence (nine 
items, a=.90, “How has your ability to use a foreign language developed in the course of the 


exchange? Please tick any that apply. e.g. ‘Ability to understand’”). Pre- and post-test Cronbach 
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Technological 
Pedagogical 
Knowledge 

(TPK) 


alphas indicated reliable constructs, and 


pre-post foreign language competence 


=_ SSS ~ 
correlations indicated a rho=.XX, p<.01. 


Technological \ 
Pedagogical Content 

Knowledge 
(TPACK) 


Technological 
Knowledge 
(TK) 


Foreign language competence development 


is inevitably contextual. Accordingly, these 


were not standardised tests, but situated and 


Technological 
Content 
Knowledge 
(TCK) 


pragmatic attempts to capture the kind of 


limited language gains learners might make in 


the course of a short-term intervention. 


At the end of the post-test, four items were 


included to measure overall satisfaction with 


p the telecollaborative exchange (four items, 


‘edagogical 

Pcie fete 
nowledge 

(PCK) 7 


C Ss 


a=.91, “I found the telecollaborative exchange 


useful for my future career as a teacher”). 


3.1.3. Data analysis 


Figure 7. The TPACK model of 


In line with recommendations by Field 
(2013), we first analysed the data for missing 
values and normality in terms of skewness and kurtosis, which was confirmed. Afterwards, 
we constructed the pre- and post-test constructs of TPACK, intercultural competence and 
foreign language competence. Subsequently, given that longitudinal data collection is sensitive 
to missing data (Huisman & Steglich, 2008; Torgerson & Torgerson, 2008), we tested whether 
there were any self-selection or non-response biases in the instruments, and we conducted the 
following steps across the two rounds of data collection; first, we compared whether there were 
differences in the control versus the experimental group at the pre-test, whereby due to the 
quasi-experimental design (i.e. not randomisation of students but randomisation across different 
classes/groups) students in the experimental condition were on average three years older, more 
likely to be female (73% vs 58%), and more likely to be in primary education specialisation (84% 
vs 38%). Nonetheless, the pre-test scores on TPACK were similar between the treatment and 


control group. 


Second, we compared whether particular groups of participants were more likely to complete a 
survey in general, and the pre-test, diaries, and post-test in particular. In total, 177 participants 


completed all six instruments in full (pre-test, four diaries, and post-test). In terms of pre-test 
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bias, students who completed the pre-test were more likely to complete the post-test, and also 
had higher post-TPACK scores. In terms of the diaries, students who had higher scores on pre-test 
TPACK and intercultural communicative competence scores, women, and older learners were 
more likely to complete the diaries. In terms of post-test bias, students who completed the post- 
test were more likely to complete the pre-test, and also had higher pre-TPACK scores and pre- 
intercultural communicative competence scores, but no difference in gender and age was found. 
In terms of completing both pre- and post-tests, students who specialised in secondary education 
were more likely to complete both surveys while students in early childhood education were 
less likely to complete both surveys, although there was no difference in terms of specialisation 


(e.g. mathematics, music, social studies). 


Third, we compared whether the response rates and results for instruments were different in 
the two rounds. In general, we found that Round 2 had slightly better response rates and slightly 
higher TPACK and intercultural communicative competence scores. We found that participants 
in Round 2 had significantly higher pre- and post-test scores on TPACK, technological pedagogical 
knowledge, and TPACK at the pre-test, and higher TPACK at the post-test, although all with a 
small effect size (see Richardson, 2011 for more details). In contrast, in terms of intercultural 
competences, participants in Round 2 had significantly lower intercultural effectiveness (overall), 
interaction relaxation, interaction management, and message skills at the pre-test, although 
again with a small effect size. Significantly more participants in Round 1 completed the pre-test, 
while significantly more participants in Round 2 completed the post-test, and overall there was 
significantly more data for pre- post-tests for Round 2 participants, again with a small effect size. 
Therefore, we argue that the two rounds are comparable and therefore we can aggregate the 


two rounds into one dataset, and thus increase the statistical power. 


Overall, across the six measurements, respondents who were female had higher TPACK and 
intercultural communicative competence scores and were not studying primary education were 
more likely to contribute to the various survey instruments. This is a common finding in social 
science research (Richardson, 2012; Ullmann et al., 2018), whereby ‘lower performing’ student 
teachers and ‘harder to reach’ students are less likely to respond to surveys, in particular when 


completing multiple surveys over time. 


Subsequently, we used ANOVA and repeated ANOVAs to test the impact of the training, using effect 
sizes to determine the impact of the respective tests (Richardson, 2011). Follow-up correlation 
and regression modelling was conducted to control for mediating factors (e.g. gender, age, 


specialisation), which are not illustrated to enhance the readability of the report. 
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3.2. Qualitative research methods 


In order to achieve a more comprehensive and perhaps nuanced understanding of the impact of 
virtual exchange in initial teacher education, the EVALUATE team also collected a large quantity 
of qualitative research data from the students and teachers participating in the exchanges. 
These qualitative data sources allowed us to achieve a comprehensive data capture and to 
answer those questions which may not be susceptible to investigation using a classic pre-test, 
post-test method. Furthermore, in line with recommendations of Rienties et al. (2013a, 2013), 
triangulating the quantitative with qualitative data was used to explain different findings as 
regards the impact of the virtual exchanges in the participating institutions and, significantly, to 
shed light on the experiences of the teacher trainers as they ran virtual exchanges in their classes 
(fourth research question). Such an approach is supported by Anderson (2008) who argues that 
“[o]ften a combination of qualitative and quantitative approaches is needed in cross-national 


research due to the highly diverse cultural contexts to be studied” (p. 91). 


The main sources of qualitative data related to the study of students’ experiences and learning 


gains were: 


pre- and post-exchange online or on-site interviews with focus groups from 


the participating classes and their teacher trainers; 


an online learner diary provided at key stages during the exchange with 
prompts focussing on the telecollaborative experience and its relationship 
to the development of digital-pedagogical competences and intercultural 


competence; 


the online interactions and products created by the students during the 


(c>:(olatebarexcear- valor 


the end of course reflection documents such as course portfolios or essays, 


and interviews with the teacher trainers who were running the virtual 


exchanges. 


In the case of the study related to the experiences of the teacher trainers (see Section 5), the main 


sources of qualitative data were: 
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written ‘teacher diaries’ during the preparation phase of their virtual 
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written email interviews during their exchanges; 


notes on critical incidents reported by the teachers or by their mentors during 


the exchanges; and 


online oral interviews via videoconferencing at the end of the virtual exchange. 


To analyse the data and help to answer our research questions, qualitative content analysis was 
used (Zhang & Wildemuth, 2009). This is a widely used qualitative research technique which 
goes beyond merely counting words, and instead carries out the subjective interpretation of the 
content of text data through the process of identifying themes or patterns and systematically 
classifying them through the use of codes. Hsieh and Shannon (2005) identify three distinct 
approaches to qualitative content analysis: conventional, directed, and summative. All three 
approaches are used to interpret meaning from the content of text data, however there are 
significant differences in the approaches. In conventional content analysis, coding categories 
are taken directly from the text data. In a directed approach, a pre-existing theory or relevant 
research findings are used as guidance for the creation of initial codes. Finally, a summative 
content analysis involves the counting and comparison of keywords which is then followed by 


the interpretation of the underlying context. 


In this study, both conventional and directed approaches to qualitative content analysis were 
used in different contexts. In the context of analysing students’ learner diaries and interviews 
to explore their experience of digital, intercultural, and foreign language learning, the research 
team took a conventional approach to content analysis and avoided using preconceived 
categories to code the data. Instead, working in teams of two researchers per question, they 
carried out an initial analysis from a portion of the data and allowed the categories and codes to 
emerge from this data. Following an evaluation of inter-coder agreement, these codes were then 


consolidated and the rest of the data was coded by the researchers. 


In the case of the data related to the experiences of the teacher trainers, a directed approach 
was used. Using existing research on the experiences and challenges of university educators 
who endeavour to introduce virtual exchange into their own teaching, two researchers began 


by identifying key concepts and themes as initial coding categories. As data was collected 
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primarily through interviews, open-ended questions were used, and these were then followed 
by targeted questions about the predetermined categories. This is described in more detail in 


Section 5. 
3.3. Data collection process 


As indicated in Figure 8, each recruited participant for a particular exchange received a 
personalised email to complete the pre-test (as described in Section 3.1 above). In the first round, 
some participants also accessed the pre-test via a generic link provided by their teacher. After 
each task, students were encouraged to reflect upon their experiences in an entry of their learner 
diary. At the end of the virtual exchange, each recruited participant received a personalised 
email to complete the post-test. Data was gathered using the Qualtrics survey tool by the Open 


University and subsequently analysed as described in Section 3.1. 


Before Task 1 and after the respective three tasks, participants were requested to share their 
experiences and lessons learned in online learner diaries. At the pre-test stage, four open 
questions were posed in the learner diary, and a total of 438 participants responded. The 
questions in this first entry of the learner diary focussed on the students’ previous experiences 


of intercultural contact and of using digital technologies in education. 


Start of the Telecollaboration End of the Telecollaboration 


End of Task 3 


End of Task 2 


End of Task 1 


Figure 8. Data collection process of the EVALUATE study 


In total, 374 participants completed the second entry of the learner diary. In this case, six open 
questions were raised in relation to their initial experiences of the virtual exchange. Following 
the completion of Task 2, 277 participants completed the third learner diary which included 


five open questions related to their evolving experiences in the project. For example, students 
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were asked “have you learned anything about your own or your partners’ culture that you 
didn’t expect?” Finally, just before the post-test, Diary 4 posed nine open questions. In total, 
385 participants completed this entry which asked them to reflect on their overall impressions 
and conclusions from their exchange (the full list of questions from all the learner diaries can 


be seen in Table 2 below). 


Table 2. Learner diary question sets 


Can you give a concrete example (a lesson, a series of lessons, or even an entire class/course) 
of how the use of technology has enhanced teaching or learning (a situation where you were 
either a learner or a teacher)? What was the topic and which tools and/or online environments 
were used? 


Have you ever used online technologies before to communicate or work with people from other 
cultures? If so, give a concrete example. 


How would you describe your cultural background? 


What do you hope to achieve or learn from this virtual exchange? 


In your first task you used different technologies — online tools — for communicating and 
presenting information. What have you learned about selecting technologies that enhance what 
you teach and how you teach? 


What have you learned from your first task about selecting technologies that enhance what 
students learn and how they learn? 


What do you feel you have learnt about your own and your partners’ national and educational 
cultures? 


Do you think you or your partners have any stereotyped views of each other’s cultures? If so, 
describe them? Have you discussed them or just noticed them? What do you think lies behind 
these stereotypes? 


After working together on this first task, what are your initial impressions of your virtual 
partners? 


Did you learn or use any new elements of language in completing this task? 


Often the way an online tool is being used changes depending on what learners are supposed to 
do while carrying out the task. What have you learned about the need to adapt online tools to 
how learners use tasks in your subject area? 


How do you feel about the interactions with your virtual partners so far? What are you finding 
easy or difficult? What steps do you think you could take to improve the interactions? 


What have you learnt about the topics that you investigated? What cultural differences and 
similarities did you notice between the way your topic is dealt with in the two contexts? 


Have you learned anything about your own or your partners’ culture that you didn’t expect? 


Did you learn or use any new elements of language in completing this task? 
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Learner Diary 4 (close to post-test) 


e What have you learned so far about how technology influences your teaching approach? 


How do you feel about working in an intercultural team? 


Describe any challenges you faced in your group work. What do you think the causes were? 
How did you try to solve them? 


Did you learn or use any new elements of language in completing this task? 


If you used a foreign language to communicate in the exchange: How (if at all) has your ability 
to use a foreign language developed in the course of the exchange? 


If your foreign language use has not improved, can you explain why the exchange didn’t help 
you in this way? 


What was the most important thing you learned from this experience? 
Is there anything that has positively affected your telecollaborative exchange experience? 


Is there anything that has negatively affected your virtual exchange experience? 
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4. Empirical findings of the EVALUATE 
study of student learning 


In the following section we present the main findings of the EVALUATE study. The results look 
first at the overall satisfaction of learners with the experience of virtual exchange before going 
on to look in detail at the outcomes of the studies which look at intercultural, digital-pedagogical, 
and foreign language learning outcomes. The section concludes by presenting the findings of the 


study of the teacher trainers who engaged their classes in virtual exchange. 
4.1. Satisfaction with virtual exchange 


At the end of the virtual exchanges, 564 participants completed the post-test survey, which 
included four questions about their overall satisfaction with their virtual exchanges. Taking a 
cut-off of 3.4 in line with Rienties, Brouwer, and Lygo-Baker (2013b), 64% of participants were 
positive overall with their virtual exchanges based upon the combined satisfaction scores of the 
four items. As indicated in Figure 9, the vast majority of participants indicated that they “learned 
a lot from the telecollaborative exchange”, with a mean score of 3.52 out of 5. Similarly, most 
participants believed the virtual exchanges will be useful for their future career, with a mean 
score of 3.81, with 75.2% of participants that would recommend other student-teachers to do 
telecollaboration, with a mean score of 3.91. Finally, 64.6% of participants indicated that they 
would like to see telecollaboration included in other teacher-education courses in their degree, 
with a mean score of 3.68. In other words, participants in general were positive to very positive 


about and satisfied with their learning experience. 


Regarding the qualitative analysis, most of the participants reported on the positive impact of 


virtual exchanges on their learning experience, as these quotes and Figure 9 below illustrate: 


“T learned a lot with this experience. It was different from any other activity that I have 
done. It help me for my communication skills, in the practice of the second language, team 


work, learning about the education in a different culture and more”. 


“T believe that the course has expanded my perspectives about teaching and made them 
become more solid and clearer than before. It was certainly an eye-opening learning 
experience that shaped my character as a teacher and taught me how to incorporate tech 


and cultures in English teaching”. 
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| would recommend other student-teachers to do telecollaboration in their | would like to see telecollaboration included in other teacher-education courses 
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Figure 9. Satisfaction with the virtual exchanges 


In this section, we explore whether participants increased their intercultural competence, 


TPACK, and foreign language acquisition. 


4.2. Impact of virtual exchange on students’ intercultural 
communicative competence: quantitative findings 


There are many definitions of intercultural communicative competence, most of which include 
affective components, knowledge, and skills (for example Byram, 1997; Chen & Starosta, 1996; 
Deardorff, 2006). These are also defined as cognitive, attitudinal, and behavioural paradigms of 


intercultural competence (Hammer, 2015). 


The quantitative tool selected for this study was developed by Portalla and Chen (2010) to 
measure one of these components, intercultural effectiveness, that is, the behavioural aspect of 
intercultural communicative competence which refers to the ability to attain communicative 


goals in intercultural interaction. Interculturally effective behaviour has been broken down into 
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several different components which scholars have identified as accounting for effectiveness. 

alla anc =n (2010) drew on some of these conceptualisations of intercultural effectiveness 
and developed a 20-item survey, based on a factor analysis of 76 items found to be important 
for intercultural effectiveness in a review of the research. In their validated intercultural 
effectiveness scale, Portalla and C 2010) identified six factors, each with their own sub- 


scales: 


* Behavioural flexibility is the ability “to observe an interaction, distinguish 
and make use of the appropriate behaviours, and adapt to the specific 
situational context” (Portalla & Chen, 2010, p. 23) and the ability to respond 
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implies an awareness of the environment in which this is taking place. 


Interaction relaxation is a factor which indexes lack of anxiety or 
apprehension in their interactions with ‘others’. As Portalla and Chen (2010) 
write, “people scoring high in the [intercultural effectiveness scale] are 
less characterized by an unpleasant emotional state, feelings of tension or 


apprehension and worry [...] towards the perceived interaction” (p. 28). 


Interaction management and message skills are more closely linked to 
language and communicative competences, the ability to express oneself and 
LUbaYoKcpusie-balommopalcmr-Daloldarcoum'salcyammooyaavanloba(er-nabat:an ‘eld almy ols10)0)(-mpeuovremmobNeicoucyate 


cultures’. 


Identity maintenance is seen as a form of facework in intercultural 
interactions, that is, the demonstration of effective behaviours to promote the 


other’s cultural identity. 


Interactant respect is other-orientation in interaction which reflects the 
recognition of the reciprocal and interdependent nature of interaction and 
is linked to relationship cultivation, that is, the ability to establish positive 


interpersonal relationships through their interactions. 


We first compared the smaller subset of the experimental and control condition in Exchanges 


15, 16, and 19. As indicated by Figure 10, the experimental condition participants (n=122) 


had slightly higher intercultural communicative competence scores before the treatment in 
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comparison to the control group (n=63). At the post-test the experimental condition (n=122) hada 
higher intercultural communicative 

competence total score relative to the 

aot pre-test, indicating a small positive 
te increase over time. When comparing 
the increase with the control group, 
although the experimental group 
did have a higher intercultural 
communicative competence score at 
the end of the virtual exchanges, this 


Pre-intercultural Post-intercultural was not statistically significant. 
communicative competence communicative competence 


=———=Experiment =< < Control 
Afterwards, we compared _ the 


Figure 10. Pre- and post-intercultural communicative intercultural communicative 
competence scores of experimental and control conditions competence developments with the 
(exchanges 15, 16, 19 only, n control=63, n experiment=122) wider group of participants, whereby 
we included all participants who 
completed both pre- and post-tests. Taking a cut-off of 3.4 in line with Rienties et al. (20135), at the 
pre-test, 77% of experimental students had positive intercultural communicative competence 
scores, in comparison to 69% of control students. At the post-test, 83% of students in the treatment 
condition had positive intercultural communicative competence scores, in comparison to 73% of 
the students in the control condition. 

—_—_ ae In general, using paired t-tests, there 

-_-- was a strong and significant increase 
in intercultural communicative 

competence over time (t=8.102, 

p<.001), though with a small effect 

size, indicating that overall the 577 

participants developed _ stronger 


(self-reported) intercultural 


Pre-intercultural Post-intercultural 


Baie aay communicative competence over 
communicative competence communicative competence 


—<—<=Experiment =< =< Control time. Nonetheless, there was no 


significant difference in terms 
Figure 11. Pre- and post-intercultural communicative of intercultural communicative 


competence scores of experimental and control conditions competence gains over time between 


(n control=63 , n experiment=516) 


the experimental and control groups. 
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As illustrated in Figure 11, although the experimental treatment group reached nearly 4 as an 
average, the students in the control condition were also more confident in their intercultural 


communicative competence skills. 


Subsequent analyses per sub-construct did highlight significant differences between the 
treatment and control group, whereby the treatment group had significant post-test scores 
on behavioural flexibility (F=5.494, p<.05), interaction management (F=5.089, p<.05), message 
skills (F=9.101, p<.01) and the combined intercultural effectiveness construct (F=4.918, p<.01), 
all with a small effect size. In other words, in terms of intercultural communicative competence 


total development, there appeared to be no significant differences in developments over time 


Pre-interaction relaxation Post-interaction relaxation Pre-behavioural flexibility Post-behavioural flexibility 


———Experiment <<< Control —— Experiment <— < Control 


Pre-interaction management Post-interaction management Pre-message skills Post-message skills 


——Experiment =—=«—<« Control — Experiment =< =< Control 


Pre-identity maintenance Post-identity maintenance Pre-interactant respect Post-interactant respect 


———Experiment —«. Control — Experiment «— =< Control 


Figure 12. Pre- and post-test comparison of subcomponents of intercultural 


communicative competence (n control=63 , n experiment=516) 
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from a quantitative perspective when compared with the control group. Nonetheless, small 
significant differences at the end of the exchanges in terms of behavioural flexibility, interaction 
management, message skills, and intercultural effectiveness were found between the treatment 
and control group (Figure 12 above). In the subsequent sections we will unpack the underlying 
reasons why some of the subcomponents of the intercultural communication skills developed 


stronger than others. 


4.3. Impact of virtual exchange on students’ intercultural 
communicative competence: qualitative findings 


Looking at the qualitative data can help us interpret the quantitative results. It was clear from 
the students’ diaries that this project presented them with some communication challenges or 
hurdles that they had to overcome. The project required them to communicate with distant peers 
in an online context and thus provided participants with opportunities for putting into practice 


the language and intercultural skills they were developing through their courses. However this 


Intercultural issues in 


Vata dUr-1m-> Corar-lare (=) 


Lack of engagement / motivation 


Bad sense of teamwork 


Gain confidence and flexibility 


Communication issues 


Lack of self-expression 


Negotiation 


Misunderstanding / Misinterpretation . 
‘ Leadership 


Decision-making 


Different work ethics 


Intercultural knowledge 


Different interpretation of assignments Improve the skills 


Different levels of autonomy 
Language 


Collaborative competence 


Technological problems 


Conflicting timetables Problem-solving 


Technical issues 
) Conflicting time zones 


Different organisation methods 


| Language isucs | issues Language barrier / poor communication 


Figure 13. Intercultural issues in virtual exchange 
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‘real life’ experience of online communication and collaboration meant they also had to face 
communication, cultural, technical, and/or linguistic issues which they may not have expected. 
In general terms, Figure 13 above presents the challenges participants faced and the skills 
developed through the project in relation to intercultural competence. The figure is based on the 
codes which were developed through analysis of qualitative data. The diagram highlights the 
complexity of virtual exchange, and the many issues participants had to face, but offers an idea 
of how quantitative and qualitative data and results complement one another. For the majority 
of participants it appears that it was actually in facing these challenges and solving problems 
that they acquired new competences related to intercultural effectiveness, competences which 
relate to behavioural flexibility, interaction management, and messaging skills, which also 
relate to digital competences (as discussed in Section 4.3) and language competence (Section 4.4). 
However, the fact that there were indeed many challenges, and students’ reflections indicated 
that not all students felt they managed to overcome these difficulties and remained frustrated at 


the end of the project, may explain the small effect size. 
4.3.1. | Behavioural flexibility and interaction management 


The scrutiny of the qualitative data provides evidence that students had to strengthen their 
communication skills to fulfil the aims of the project. At the beginning of the virtual exchanges, 
though student expectations about the exchange were high and there was generally a positive 
feeling towards the exchange, in their later reflections a substantial number of students 
commented on the anxiety and difficulties they had initially faced. As illustrated in Figure 13, 
the challenges they faced included communication and language issues: what they perceived 
as a lack of engagement on the part of their peers, difficulties in working as a team, difficulty in 
expressing themselves, and/or in understanding their ‘unknown’ international peers. Some of 
the participants ascribed the difficulties they were having to cultural issues, such as different 
work ethics, different interpretations of assignments, and different levels of autonomy. Other 
issues were of a technical or logistic nature, such as difficulties with the tools they were using, 


negotiating which tools to use, time zones, and student timetables. 


However, at the end of the programme, many of these students reported that they felt more 


confident in working in an international and intercultural environment: 


“T’ve learned to overcome my fear of not being understood by the people from different 
countries. I tried to formulate my utterances as clearly as possible, and it worked. At first 


I was scared, but then not really. Our communication was great!”. 
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In addition, participants’ accounts of their experiences demonstrate their increasing willingness 
to understand their exchange partners’ ways of thinking and working, and adapt to specific 


contexts: 


“Tt is strange at first because you can tell that your virtual partners have a different way of 
doing things and this is unsettling. However, as time goes by, this helps you to concentrate 
a lot better and to look for answers and solutions that both sides are happy with. At the 
same time you also make use of English and get the feeling that you have learned about 


aspects of another culture that you didn’t know before”. 


In cases such as this, students seemed to develop behavioural flexibility as they tried to internalise 
partners’ ways of thinking in order to come up with different ways of accomplishing the tasks. 
For some students this process was also seen to influence their self-perceived identities which 


are negotiated through the course of the interactions: 


“T learned a lot about myself and about my Mexican and Israeli exchange partners. It 
had a profound effect on me personally, socially and culturally because I found out what 
they thought about education and the way they work (which is very good indeed). I also 


learned more about their culture”. 


A limitation of the intercultural effectiveness scale is that it was designed — like many tools 
measuring intercultural communication skills - to consider effectiveness in face to face 
interactions, not online interactions, with the additional layers of complexity that the medium of 
technology may introduce. Online spaces are quite different, but flexibility is equally important. 
A further type of behavioural flexibility we found evidence of was students’ adapting their 
communication strategies for their specific online contexts. This often entailed resorting to 
different modes and tools for communication in order to address the communication problems 
they were having with their peers (as also discussed in Section 4.3). Many students reported 
choosing tools which they were more familiar with (many cited WhatsApp) or they felt were 
more appropriate for communication with their peers than the learning management platforms 
used in class, and this allowed them to communicate more authentically and flexibly, as also 


reported in the section below on digital competences. 


“Thad difficulty communicating through Uniko because of technical problems. My group 
members and I solved it by texting in another social network: WhatsApp. In addition, it 


was challenging to agree about the lesson rundown, the digital tools and activities [we] 
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used in the project because of cultural gaps. We solved by listening to each other and 


compromis[ing]”. 


Another issue in measuring intercultural effectiveness using scales such as this is the 
assumption that an increase in perceived effectiveness indicates a positive result. Yet in practice 
intercultural communication and collaboration is often more complex than one might expect 
it to be (Jindal-Snape & Rienties, 2016; Volet & Jones, 2012). In line with Kruger and Dunning’s 
(1999) and Dunning’s (2011) effect, a reduction or limited change in students’ self-assessment of 
this component may actually indicate greater awareness of the complexity of the process once 
students have actually experienced it rather than the initial, perhaps ‘idealised’, conception of 


intercultural communication and their ability to effectively communicate in such a context. 
4.3.2. Experience and engagement with difference 


As reported above in the description of our approach (Section 3.2), we analysed the qualitative 
data using a conventional content analysis, which has affinities with a grounded theory approach 
as we did not initially use any predefined categories from the literature on intercultural 
communicative competence. However, some of the questions participants were asked to reflect 
on in their reflective diaries were clearly related to intercultural issues, and after our initial 


coding of the data we made links with the research literature to support and explain our findings. 


Intercultural communication is generally considered to be about engaging with difference 
(ethnic, national, religious...) and indeed the word ‘different’ was the most common term in the 
dataset we analysed. Within this theme of difference we developed a range of sub-categories 
that identify how and why participants engaged with difference. The students’ reflections on 
difference, and the extent to which their engagement with difference leads to a questioning 
of their own identities and beliefs, also known as ‘distancing’ (Cummins & Sayers, 1995), can 
help us understand the participants’ levels of intercultural understanding and development. It is 
important, however, to also look at the participants’ starting points, that is, how much they had 
experienced and reflected on intercultural interactions prior to their virtual exchanges, as the 


progress they make is likely to depend on this. 
4.3.3. Participants’ descriptions of their cultural backgrounds 


In the first diary, which was completed before taking part in the virtual exchanges, participants 


were asked to write about their own cultural background (How would you describe your cultural 
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background?). Responses to this question, which was deliberately open, provide us with some 
insights into respondents’ intercultural awareness, the extent to which they view identity and 
background as a complex, fluid construct, or as a static, national, and perhaps essentialist notion. 
Responses to this question also gave us a sense of the extent to which respondents’ may have had 


experience of intercultural encounters. 


The question was interpreted in various ways. Cultural background is here mostly understood 
as having an identity which is linked to nationality, language, religion, a particular social 
class, or being located in or having the culture of a specific region, such as ‘American’, 
‘Western’, or ‘European’. In our coding of the data, three broad categories emerged in relation 
to respondents’ descriptions of 
mam Number of students 

their cultural background: 
homogeneous environment, 
multicultural environment, and a 
third category, complex identity 
which was used for those responses 
where a more complex view of 
cultural background and identity 
was expressed. The majority of 
respondents reported coming 
from or living in homogeneous 


environments, a substantial 


Multicultural environment 
cultural background 


Homogeneous environment Complex identity 


number in multicultural 
environments, and a fraction of 
Figure 14. How would you describe your cultural background? them had a more complex view of 
cultural background and identity, 
and saw themselves as being in a process of identity construction, where all experiences could 


have an influence on them (see Figure 14). 


The notion of ‘homogeneous environment’ relates to an understanding of someone who sees 
him/herself as part of a homogeneous community, which might be quite diverse, but there is 
mention of a broader national unity which gathers together differing parts. In other words, the 


notion of identity is bound together with that of nationality. 


“T am fully German. I was born in Germany my parents are German and so is the rest of 


my family”. 
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“T am lucky with my culture and cultural background. Almost all my family comes from 
Spain, but from different parts, so I know many cultural aspects of different territories in 


Spain”. 


The second example expands the idea of being Spanish to embrace unity in diversity, as the 
respondent mentions different cultural aspects within the Spanish territory, but there is still 
homogeneity in the sense of being part of a Spanish national identity. ‘Homogeneous’ also means 
restricted in terms of engaging with difference, e.g. “I grew up in Limhamn, a suburb outside of 
Malmo which almost entirely consists of ahomogeneous population. In other words, my cultural 


background is not diverse”. 


By contrast, we applied the category ‘multicultural environment’ when participants referred 
to their background as multicultural or they claimed to be multilingual. This category includes 
people who have grown up in relatively homogeneous contexts, but who report experiences 
of living abroad and/or studying other cultures and learning foreign languages which they say 


have impacted the ways in which they interact and see other cultures. 


“As I am from the Netherlands I can describe my background [as] multicultural. I have 
friends from all different cultures over the world, and for me it is really easy to connect 


to them”. 


“IT am Polish woman, a Catholic. I grew up in the centre of Poland and now I am living in 
Warsaw. I speak some languages. Besides Polish and English I know French very well, 


I know a little bit of Spanish, Russian and Latin. This year I started learning Swahili”. 


The responses do not suggest that this contact with other cultures has had any impact on the 
respondents’ own identities or offered them different perspectives, and there are hints of 
ethnocentrism and ‘othering’ in some of the comments, with a common pattern of ‘us’ versus 


‘them’. 


In the third category, ‘complex identity’, respondents put into question their own identity and/or 
see it as a symbiosis of diverse experiences they had in other countries and cultures. They report 
an experience that has a long-lasting effect on the ways in which they see and perceive themselves 
and the world around them. Here participants see themselves as bi-multi-lingual/bi-multi-cultural, 
e.g. “I grew up in Greece though having Polish parents. My personality has been definitely affected 
by this fact. What is more, I am bilingual (Greek & Polish). I believe that these facts made me more 
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open-minded”; Europeans, e.g. “I am first of all European, then Polish. I don’t identify all that much 
with the Polish culture when set against the rest of Europe, but I’ve noticed that when faced with 
more distant cultures (East Asian, American) I do identify as European”; international citizens, 
e.g. “I am German but I have always been traveling a lot. I would therefore describe myself as an 
international person who is very interested in the cultures of other countries”; or they might have 


a multicultural mindset, such as the following quote: 


“T grew up in Sweden, but I’m originally (and born) in Croatia. So, I grew up in two different 
cultures which shaped me a lot as a person. I did an exchange in Macau, where I met and 
befriended a lot of friends from a lot of different countries and continents. Which again, 
shaped me even more. It makes you think, understand differently. Because, I felt like I was 
very open-minded and understanding before because of my background, and I was but 
not really, because you never truly understand, and it’s something that’s very hard to 
explain. Your thinking completely changes. But, when interacting and listening to others’ 
thinking, who have different backgrounds from yourself, it just, changes you. I definitely 
got a huge reversed culture shock when I came back to Sweden. It was hard to re-adapt 


with a new you”. 


4.34. Participants’ engagement with difference through virtual exchanges 


During the virtual exchanges, participants engaged with difference on multiple levels. Indeed, 
‘different’ was the most frequent word in the data we collected, as can be seen from the NVivo 
word cloud (Figure 15 below). People and students were also among the most frequently used 


terms, highlighting the human dimension of the participants’ virtual exchange experience. 


We identified five levels of engagement with difference in their reflections, which we describe 


below: 


(1) No difference. Participants see no significant difference between their 
national and educational culture and that of their international peers: “even 
coming from other countries, we are all the same. We all have obstacles, jobs 


and other challenges in life, but we are all looking for the same thing, which is 


to improve ourselves as professionals in the field of education”. 


(2) Minimising difference (‘we are all the same after all’). Participants are 


aware of cultural differences, but do not scrutinise them, showing a low level 
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of reflexivity regarding this issue: “I learnt that our cultures do not differ 
much. Of course, we live in different countries so our attitudes may vary 
however, thanks to the globalisation it is possible to know many things about 
other cultures. Also, Spain and Poland are both members of the EU so there is 


oy aalem .@baXe Moympavoyantoy:xcvapecyela(oyame)imuatsm=roleCersla lo) at. me) Kole sian 


(3) Exploring different perspectives on education. Students compare their 
educational systems and reflect on how the differences and similarities might 
shape their own views on education as well as their expectations and the 
challenges they faced in the exchange itself: “we used the platform to talk 
about the education system and the schools that they have there. We compared 
these with our schools and saw the differences. We then asked questions 
and continued on in this way to keep the conversation going”, as well as the 
following: “students like us, have similar life styles but in some aspects seem to 
have a different idea of cooperation work. This might result from their lecture 
being organized differently than ours and therefore us having expectations 
they weren’t told to fulfill”. In these extracts there is an engagement with 
difference beyond static national culture categories, but there is an us-them 
orientation, with respondents’ own culture being the standard by which 


others are measured. 


(4) Deeper engagement with difference. Participants compare their cultures, 
JateV oy Luspur-DaLo Un A\cchycume) mdapvalobatcmr-valoljopumols) atcarabalcmmelsvaelevakjou:lovalcar- my avedalsoum (arc 


of self-reflexivity and openness towards their foreign peers: “I learned that 


we have a different way of thinking and acting. This does not mean that one 


is better than the other, just that they are different and I think it is really 
interesting to be able to learn and even experience the habits and ways of life, 
which change depending on what we are used to doing in our everyday lives”. 
The use of inclusive ‘we’ in the reflection above suggests that the participant 
is also reflecting on the cultural embeddedness of their own beliefs and values 


and how one’s life experiences also affect these beliefs. 


(5) Seeing complexity in diversity. Participants reflect on the complexity 
that exists between different cultures and also within their own in a 
more abstract/theoretical way: “I think this experience will help us all to 


understand a little bit more about educational backgrounds around the 
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world and how the culture of each country plays a fundamental role in the 


language learning-teaching process”. There is an understanding of the many 


different dimensions of cultures, identities and beliefs, and of the factors 
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Figure 15. Representation of word frequency in the intercultural study 


These different levels of engagement echo Bennett's (1986, 1993) developmental model of 
intercultural sensitivity, which describes different ways people react to cultural difference. In 
Bennett’s (1986, 1993) model, there is a progression along a scale from ethnocentric positionings 


of denial, defence, and minimisation of difference to ethnorelative attitudes of acceptance, 
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adaptation to, and integration of difference. This model is problematic in several respects, for 
example in the conceptualisation of intercultural sensitivity as a scalar, linear phenomenon, and 
in assuming the monocultural learner as the starting point (Liddicoat, Papademetre, Scarino, & 
Kohler, 2003). Furthermore, his, and other studies of the development of intercultural sensitivity 
and awareness (Hammer, 2015), have been developed for and extensively explored in the context 
of study abroad, and have rarely been applied in virtual exchange contexts. What our analysis 
of the qualitative data has found is that the levels of intercultural sensitivity identified in the 
data bear some similarities to the positionings in Bennett’s (1986, 1993) model, and development 
of intercultural sensitivity through virtual exchanges is in some ways similar to that which can 
emerge in some study abroad experiences. To reach a level of critical engagement with the 
complexity of intercultural communication is not easy through virtual exchanges (Helm, 2017) 
— nor indeed is it guaranteed through physical mobility (jackson, 2019). It also depends very 
much on the starting point of the participants. However, through carefully planned intercultural 


interventions it is possible (Jackson, 2019). 


Within the context of this project, where a majority of participants started with limited 
intercultural experiences and awareness as well as a degree of anxiety towards the exchange, 
minimisation of difference was the most common attitude we found in the data from the end 
of the project. However, in some participants we found evidence of a shift in mindset from 
largely ethnocentric positionings to a realisation that there are other perspectives on education 
and related issues. This is an important step as it can be the trigger for further exploration and 


learning: 


“At the first time, I was a bit afraid because I did not know how this worked. However, 
after the first contact, I had the necessity of knowing more about their culture and the 
way their [sic] study in their country, because, as we know, not all the schools and colleges 
have the same methodology. Now, it is a pity that this experience is almost over, but I hope 


I could participate in another telecollaboration soon”. 
4.3.5. Triggers and barriers for engagement with difference 


If we are exploring the development of intercultural understanding, it is important for us to 
understand what triggers participants’ deeper engagement with difference and their developing 
greater understanding of their own language(s) and culture(s) in relation to others. As discussed 
above, in some cases, participants’ entered the project with what we described as a multicultural 


background. In a small number of cases this led to a dismissal of the virtual exchanges as a 
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banal experience with low levels of intercultural engagement, whilst in other cases this led to 
greater reflexivity on the exchange itself, how this differed from other experiences they had 
had and considerations of how they could further engage with their partners: “I would say that 
this experience overall had a positive effect on my learning. This project was quite different 
from what I have been going through already in my education, so doing this unconventional 
thing broadened my views and developed my abilities”. A different participant said: “I learned 
a lot about compromise and different approaches to teaching. We often reached a point where 
we could not find common ground so we had to make adjustments a lot. They have different 


approaches to teaching English and it was difficult from time to time”. 


The following quote illustrates a high level of intercultural engagement when the participant 
challenges stereotypical views on different cultures and reflects on the value of virtual exchanges 


for the acquisition of intercultural competence: 


“Everybody have a stereotype about other cultures, it is normal when you only know 
something about other country watching tv or reading some remarkable news. But the 
point is to have a deepest knowledge about a culture or country and break this stereotype. 
The best way to do that is travelling, then you realised we all are more similar than we 
think. Other way is to keep in touch with people from abroad and this is our opportunity 
with this program. With my friend Tamara in my group we have discussed a lot about how 
different or similar we are, and how could be their education system, how can we take 
advantage of them... etc. In my opinion this a really worth task because Iam convinced ’m 


learning a lot, and I don’t have this feeling with other tasks in our degree”. 
Shared histories 


Clearly there are factors within the exchanges themselves which may lead to deeper engagement, 
and we identified several possible triggers for greater curiosity. Exchanges which involved 
countries with historic links between them, in particular shared histories of colonialism or 
genocide as in the case of Brazil and Portugal, or Israel and Germany, seem to have led to deep 
engagement and reflection on the part of participants. This could be because there was some 
shared knowledge but also different experiences and feelings towards historic events which 
led to a more strongly felt need to engage with one another and explore others’ perspectives — 
even though the shared histories were not explicitly addressed. In exchanges where there was 
a strongly perceived, tangible cultural difference, students’ curiosity was also high. Reflecting 


on each other’s culture, a Brazilian student said: “actually, we are similar due to Brazil being a 
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country that was colonised by Portugal. The curiosity is the most important thing to explore the 


habits from other locales”. 
Contextual factors 


It was beyond the aims of this study to explore in detail the differences between each individual 
exchange, but clearly there were contextual factors which impacted the levels of intercultural 
learning and engagement with difference on the part of the students (see Case Study 1 for the 
description of an exchange which led to strong intercultural learning). Though the teachers in 
this project all adopted a similar model of virtual exchange, the extent to which students had 
time to critically explore the intercultural dimension of their exchange and the nature of the 
discussions and reflections they had in class are likely to be important factors in the development 
of intercultural sensitivity and awareness. There were clearly differences in the extent to which 
the teachers themselves focussed on intercultural learning with their respective classes, the 
time allotted to and the degree of reflection on the interactions the students were having with 
their partner classes, the types of content they were discussing, and how they interpreted and 


addressed the challenges they may have been facing. 
Language as a barrier 


We found that language was in some cases a barrier to deeper engagement, for example students 
not having the linguistic competence to ask meaningful questions or respond to questions they 
were asked in a complex way (see Section 4.4). In exchanges where students in both classes were 
using their first or main language of communication, there appeared to be more of a focus on 
intercultural understanding and reflection from our analysis of the diaries (see also the first case 


study later in this study). 
Too much task-orientation? 


A further barrier to deeper engagement was different degrees of task-orientation. Some 
participants’ partners had an almost exclusive focus on tasks, with limited time for more social, 


interpersonal interactions: 


“The difficulties lie in the tasks. I feel, our German group has a different attitude towards 
completing tasks on time and thoughtfully than our partners. We also seem to understand 


the task differently sometimes. The forum posts then vary a lot and it takes some time 
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to get the conversation directed to discussing what we’re meant to discuss. When our 
conversations get going they are often interesting, though we ask more about our partners’ 


culture than the other way around”. 


In some cases participants saw this as a cultural issue which impacted their partner students’ 
behaviours, and our data suggested that many students in some classes had a strong wish for 
social interactions and relationship building which partner classes did not always reciprocate 
or feel necessary. Others perceived it as teachers’ different orientations to the exchange, such as 


the following quote: 


“Challenges were that sometimes we had different instructions from our lecturers so 
clarifying what we had to do took up quite a long time sometimes. To solve the problem 


we talked to the lecturers”. 
Importance of reflection 


Finally, it is important to also consider the limitations of the research approach we adopted. In 
order to encourage students to be honest in their reflections on the exchange and also in-class 
activities it was decided that the diaries would be anonymised and would be gathered by the 
research team rather than the teachers themselves. This meant, however, that the teachers did 
not have direct access to the students’ reflections (in some cases the teachers asked students to 
share their reflections with them but the students could choose whether to share these same 
diaries with the teachers or edit them). This could have led to a lack of engagement on the part 
of the students in the completion of the diaries, since these were not directly linked to their 
course requirements. Furthermore, there could have been missed opportunities for discussion 
and learning in the class, as some of the teachers may not have been aware of the students’ 
experiences. Reflection is a key component of the learning and meaning making process and is 
particularly important for the type of experiential learning that virtual exchange offers (Kolb, 


2015; Mezirow, 1991). 
4.3.6. Implications 


Our findings suggest that the intercultural learning opportunities which virtual exchange 
offers future teachers, particularly those who have had limited opportunities for intercultural 
interactions, are important because they are likely to be working in contexts which are quite 


different from the educational context they experienced when at school, given the increase 
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in immigration in Europe in recent years. It is fundamental for future teachers to be open to 
engaging with difference and that they develop intercultural awareness and understanding as 
they prepare to become teachers in multilingual and multicultural contexts. Ideally they should 
also develop an understanding of the complexity of identities and culture, an openness and 
curiosity to difference, rather than essentialist conceptualisations of national cultures. Given 
the limited number of future teachers who experience study abroad, virtual exchange is an 
important way of providing that initial contact which may spark their curiosity and willingness 
to seek contact and further engagement with those that are ‘different’ from them. However, it 
is important that if the aim of an exchange is to develop students’ intercultural sensitivity, then 


teacher trainers should be equipped for this and they should dedicate time to this in class. 


4.4. Impact of virtual exchange on students’ digital- 
pedagogical competence: quantitative findings 


In order to compare the TPACK development over time, we first compared the smaller subset of 
the experimental and control condition in Exchanges 15, 16, and 19. As indicated by Figure 16 
below, the control group (n=77) had a slightly higher score at the TPACK pre-test, while at the post- 
test the experimental condition (n=127) had a slightly higher score. The average learning gain 
for the experimental group was 0.30 (SD=0.50), while the average learning gain for the control 
condition was 0.18 (SD=0.51), although this effect was not statistically significant. In other words, 
over the (on average) 65 days of the exchange, both the experimental and control group students 
increased their TPACK total scores as ¥ 


they were learning in their context, 359 


but there was a slightly higher effect 3.8 
3.7 
3.6 


3.5 
Afterwards, we compared the TPACK 3 » 


for the experimental condition. 


total developments with the wider 33 
group of participants, whereby ~°< 
we included all participants who 

completed both pre- and post-tests. Pre-TPACK Post-TPACK 
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indicated having positive TPACK Figure 16. Pre- and post-TPACK scores of experimental and 
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of students in the treatment condition indicated having positive intercultural communicative 
competence scores, in comparison to 80% of control students. In general, using paired t-tests there 
was a strong and significant increase 
in TPACK over time (t=13.447, 
p<.001), indicating that overall the 


579 participants developed stronger 


(self-reported) TPACK over time. As 
34 indicated in Figure 17, in the wider 
3.3 group, the TPACK scores at the pre- 
3.2 test was slightly lower at the pre-test 
in comparison to the control group 


Pre-TPACK Post-TPACK (although not significant). At the post- 


test the TPACK scores of those who 


Experiment «Control 


received treatment was higher than 
Figure 17. Pre- and post-TPACK scores of experimental and those in the control group, although 


control condition (n control=63 , n experiment=516) only marginally significant at p<.07. 
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Figure 18. Pre- and post-test comparison of subcomponents of TPACK 


(n control=63 , n experiment=516) 
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Subsequent analyses per sub-construct did highlight significant differences between the 
treatment and control group, whereby the treatment group had significant higher gains in 
technology knowledge (F=8.235, p<.001), technological pedagogical knowledge (F=5.720, p<.05), 
and TPACK (F=3.692, p<.05), all with a small effect size. In other words, in terms of TPACK total 
development, there appeared to be no significant differences in developments over time from 
a quantitative perspective when comparing to the control group (Figure 18 above). Small 
significant differences in technology knowledge, technological pedagogical knowledge, and 
TPACK do give a suggestion that these pre-service teachers developed a stronger understanding 
of the complexities of teaching online. In the subsequent sections, we will unpack the underlying 


reasons why some of the subcomponents of the TPACK skills developed stronger than others. 
4.5. Impact of virtual exchange on students’ 
digital-pedagogical competence: 
qualitative findings 
How do these findings map against the insights gained from the qualitative data? Based on a 
qualitative content analysis (see Section 3.2) the following themes emerged from the entries in 
the learner diaries: 
icsrel abate) loy-atmutsyere k 
challenges encountered when using technology for teaching; 


most important insights gained in terms of technology used; 


experienced benefit of technology use (self); 
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The following contains further information about each theme and representative examples 


from the learner diary entries. 


4.5.1. Technology used 


This word cloud, as illustrated in , reflects the tools and applications recurrent in the data 


and how often they were mentioned (tools and apps in large font and at the center of the cloud 
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Figure 19. Word cloud of the tools and applications mentioned in the TPACK data analysis made with 


the word cloud facility within NVivo 
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appeared more often than other items). ‘Point’ stems from ‘PowerPoint’, frequently used by the 
student teachers in their diary entries. Google or Google Docs, for example appeared very often 
which underlines the students’ positive collaborative experience, since many virtual exchanges 
used Google Docs to have participants jointly create texts or tasks in their international teams, a 


tool which —- somewhat surprisingly — many of them had not known before the exchange. 
4.5.2. Challenges encountered when using technology for teaching 


There was a substantial amount of reflective comments on issues referring to technology use 
during the virtual exchanges. These were mainly due to a lack of familiarity with bespoke tools 
and applications the student teachers were asked to trial, and challenges with bandwidth and 


online connectivity. 


For example, when student teachers experienced technical problems with their learning 
platform, they moved to social media: “I had difficulty communicating through Uniko because 
of technical problems. My group members and I solved it by texting in another social network: 


WhatsApp”. 


Although these were experienced from a learners’ point of view (for ‘experiential modeling’ 
see further down), the student teachers mostly embraced the challenges and made numerous 
suggestions as to how they solved the problems and/or how they would go about solving them 


in the future. 
4.5.3. Experienced benefit of technology use (self) 


The student teachers regularly mentioned the concrete benefit experienced from using 


technology for themselves: 


“Our first task was useful because now I understand that video chat is important in 


teaching. It improves our speaking skills”. 


But when student teachers only mentioned that they had used an online tool or application 
without explaining how they benefited from its use, this reference was not included in this theme 
but in the theme ‘Technology used’ (e.g. “So far, ’ve used YouTube, Google Docs and Prezi”). 
Examples for the theme ‘Experienced benefit of technology use (self)’ include reflections that go 


beyond considerations of a purely pedagogical nature. They include perceived advantages of a 
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tool or application on a personal level provided they were linked to the tasks carried out during 


the virtual exchange. The following diary entry illustrates this: 


“In my first task I created an ‘About Me’ presentation from an online website that 
allowed me to display information about myself. I have learned that it’s important to 
select technologies that appropriately reflect you as a person, especially when creating a 
presentation about yourself. When picking which online tool to use to complete the first 
task I decided to use the ‘About Me’ website because it allowed me to accurately reflect 


who Iam as a person”. 


There was often overlap between putting into practice the affordance(s) of a tool while carrying 
out a task (i.e. ‘Methodological use of tools’) and - as a result - becoming fully aware of its 


affordance(s), and an ‘Experienced benefit of technology use’. Here is an example: 


“T have learned that technology can help you in your teaching approach. We used Google 


Docs to make a picture book”. 


Student teachers experienced the benefit that tools can help them while teaching, in this case 
using Google Docs to create a picture book together with their partner. At the same time, they 


learned how to put a specific tool, i.e. Google Docs, to methodological use. 


It became apparent that the virtual exchange element in their initial teacher education 
programme allowed the student teachers to go through the following cycle of exploratory 
practise which draws on Allwright and Hanks’s (2009) concept of exploratory practice (see 


Figure 20 below). 


Using specific tools when working on the tasks in the virtual exchange made student teachers 
realise the affordances of such tools (see Google Docs example above). The reflection phases 
in the blended learning environment of the virtual exchange facilitated this process, making 
students aware of the interrelationship between using a specific tool such as Google Docs to 
create a text together, thus facilitating the creation of the product the task instructions asked 
for. Experiencing the benefit of the tool and becoming aware of its affordance in a pedagogical 


setting will allow student teachers to integrate technology into their future teaching. 


Thus, they often mentioned the use of a tool for a specific purpose even though they have not 


implemented it in their teaching practice yet. They had, however, found out about its potential 
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through the virtual exchange they took part in (‘experiential modeling’). As this student teacher 
highlights: “My group created a Prezi presentation. It is a presentation program that allows you 
to explore and share ideas about a topic, in our case about Leon. I did not know this program 
before so it was a great discovery”. Such future orientated reflections were captured under the 


next theme. 


4.5.4. Projected benefit of technology use (self) 


The student teachers’ reflections under this theme are about how they will use technology trialed 
and tested during the virtual exchange in their future teaching practice, and what the added 
value is in terms of their didactic approaches. Hence, this theme looks at the future benefit of 
tools used in the virtual exchange from the teachers’ perspective, such as communicating with 
other teachers or how to research or present material more efficiently. Here is an example from 


a student teacher working with a new tool: 


“J didn’t use prezi before. I thought the platform is hard to work with, so I had not used it, 


but when we started to work on the task, I decided to make an effort and try it out. So our 
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group figured out how to use it. I will definitely use it as a teacher because it offers variety 


and helps creating interesting presentations”. 


Thus s/he explains how the ability to create more engaging presentations will make her/him a 


‘better’ teacher. 
4.5.5. Projected benefit of technology use (students) 


Student teachers also need to learn which tools will best support their future students’ 
competence development. Facilitating interaction and discussions in class is one important field 


as this comment highlights: 


“T think that online debating tools such as Padlet can be used to start a discussion because 
they allow students to put their arguments/answers to words first before discussing them 


plenary in class”. 


Consequently, the references of this theme are mainly reflecting the student teachers’ 
considerations as to how they will use technology trialed and tested during the virtual exchange 
in their future teaching practice to improve their students’ learning process — in terms of 
motivation, for example — and thus support them in becoming ‘better’ learners, such as in this 


student teacher’s comment: 


“Technology can be a very useful tool. It can enhance the learning experience by helping 
teachers use class time for practice and by motivating students. In Task 3, my group decided 
to work with fandom and include technology, which facilitated students’ communication 


with their peers”. 


This also illustrates student teacher’s gain in pedagogical content knowledge when working with 


technology inside and outside the classroom. 
4.5.6. Methodological use of tools 


As a result of the virtual exchange, students showed that they know how to choose a tool for 
a specific purpose or function based on tool affordances, for example, by selecting a bespoke 
tool to develop a specific competence (e.g. using Google Docs to collaboratively create, edit, and 


comment on a text). 
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4.5.7. Technology chosen to enhance teaching/learning 


Here the diary entries highlight student teachers’ learning process as to their pedagogical 
knowledge of working with bespoke tools and applications. In extension of ‘Projected benefit of 
technology use (students)’, references here clearly show which technical tool they would use for 
a specific pedagogical intervention in the classroom. A few of them showed this competence at 
the start of the virtual exchange, either because they had already used technology in a teaching 


context, or they had observed another teacher doing so: 


“As a teacher, in a unit regarding Multicultural England, I used a platform such as 
SymbalooEdu to give information and display fun resources about the theme to the 
students. I also used this platform to make webquests, fun activities, which were used 
by the students at home. To evaluate this unit and others, beside the paper examination, 
requested by the school, I have also used Kahoot and the response was really positive. The 


results were above average in the majority of the cases”. 
4.5.8. | Most important insights gained in terms of technology use 


Many student teachers commented on the new tools they had encountered and had learned 
to use in their exchange, such as Google Docs, Padlet, Zoom, or others (see ‘Technology chosen 
to enhance teaching/learning’). They also pointed out the impact this work has had on their 
pedagogical approach when thinking about their future learners and — as this student explains 
— discovering methodological opportunities in the use of new tools they had not been aware of 


before: 


“T used to think that using a PowerPoint presentation with images and colors and a Kahoot 
activity now and then was ‘innovating’ in the [English as a second language] classroom. 
Thanks to this exchange I have not only discovered new tools, but reflected about them 
and applied them into a task that could be perfectly used in a real class. For example, I did 
not think that an exchange like the one we had been engaged in would be carried out this 
way. I thought that it would be like a Skype conversation or sending emails to a group of 
people. However, thanks to Task 3 [collaboration on technologically-based task design], 
I have seen how learners can actively use their L2 to think critically even if their English is 
not at its higher level by giving them materials they can understand and a challenge that 


engages them”. 


more learner-centred focus: 


or learn, and so on”. 


Figure 21. Experienced benefit of technology use (self) 
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Thus, as a result of experiencing virtual exchange in his/her initial teacher education programme, 
this student teacher was able to integrate new tools appropriately into the task design for a 
lesson. In addition, the following comment highlights the realisation that one’s teaching 


approach changes through the integration of technical tools, from a more teacher-centred to a 


“Incorporating technological tools into our lessons changes the way we teach. As a 
teacher, I can choose which tools I want to use in my lessons, but apart from PowerPoint 
presentations, most tools would shift the focus from the teacher to the students. Using 
technology forces me to think more about the learning process I want my students to go 


through, which skills I want them to practice or acquire, what language I want them to use 


Before, during, and after the exchange, student teachers had to answer questions (see above) 
four times in their diaries. To understand student teachers’ competence development during 
the virtual exchange projects, we now compare their responses in Diary 1 before the exchange, 


and those in Diary 4 for the different themes, e.g. ‘Experienced benefit of technology use (self)’. 


The numbers on the left hand side in 
each graph represent the overall number 
of references for each theme across 
all exchanges. Hence, the numbers do 
not represent single words, but parts 
of sentences or a number of sentences, 
depending on how much of the content of 
a response relates to the specific theme, 
such as ‘Experienced benefit of technology 
use (self)’. For example, in Figure 21, out 
of all the responses from all exchanges, 
we found approximately 690 references 
in Diary 4 to the ‘Experienced benefit of 
technology use (self). The references 
we selected were positive since we had 
to show student teachers’ competence 


development. There were also some 


negative references where student teachers, for example, wrote that they did not get to know any 


new tools, but in relation to the positive references those numbers were very low and negligible in 


the overall picture. The visual information 
in the bar charts is underpinned with 


brief verbal explanations. 


The clear increase in references in 
the themes for ‘Experienced benefit of 
technology use (self and ‘Projected 
benefit of technology use (self) 
(Figure 22) suggests that the so-called 
experiential modeling approach (Hoven, 
2006) which was chosen for the design of 
virtual exchange - especially in the task 
sequences — has had a positive impact. In 
experiential modeling, online tools and 
processes teachers are expected to use 
in their future teaching are experienced 


from a learner’s point of view. In many 
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exchanges, for example, student teachers worked with Google Docs to collaboratively create 


texts with their international partners. When they then had to design technology-based tasks 


for school contexts, they often had their future students work with Google Docs as well because 


they had experienced it very positively in 
terms of collaboration. This also explains 
the increase in references for ‘Projected 
benefit of technology use (school students)’ 


from Diary 1 to Diary 4 (Figure 23). 


The differences between the number of 
references for both ‘Projected benefit of 
technology use (self) and for ‘Projected 
benefit of technology use (school 
students)’ at the beginning and at the 
end of the virtual exchanges (Diary 1 
versus Diary 4), are, in fact, startling. 
They underscore the positive influence 
of virtual exchange on awareness of 


and attitude towards technology use for 


Diary 1 Diary 4 
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Diary 1 Diary 4 


Diary 1 Diary 4 


Figure 25. Challenges encountered when using technology 


for teaching 


formal educational purposes. This can be 
explained by what is referred to in the 
literature as the ‘double mediation’ effect 
(e.g. Kurek & Hauck, 2014): in virtual 
exchange, the processes the student 
teachers and subsequently their learners 
are involved in are at least mediated 
twice: by the technology used and — in the 
majority of cases — by the use of a second 


or additional language, or a lingua franca. 


. Hence, virtual exchanges do provide 


the ideal set-up for fostering digital 


| competence development in general 


and ___ digital-pedagogical competence 
development in the context of initial 
teacher education in particular. The 
findings in relation to ‘Methodological 
use of tools’ speak to the same point 


(Figure 24). 


The qualitative data corroborates the 
findings from the quantitative data 
analysis, namely that the treatment group 
had - among other TPACK components — 
higher gains in technological pedagogical 
knowledge. The increase in references 
found as ‘Methodological use of tools’ is 
not as dramatic as for other themes (see 
above). A possible reason is the fact that 
it takes time and therefore involvement 


in more than one virtual exchange before 


substantial knowledge and competence in methodological use of online tools, and thus their 


pedagogically informed application is acquired. Yet there was clearly some gain in insights 


to this effect during the virtual exchanges which took place during the EVALUATE project 


which points to the added value of integrating virtual exchanges into initial teacher education 


provisions (Figure 25). 
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Here the difference between the number of references in Diary 1 and Diary 4 is most pronounced. 
While, at first glance, one might conclude that the learning process in terms of technology 
use during the project turned increasingly more problematic, a closer look at the reflective 
comments coded in Diary 4’s entries shows that the students have become increasingly aware of 
the challenges associated with online environments, tools, and applications, and also have had 


an opportunity through the training to learn how to deal with them. 


Small significant differences in technology knowledge, technological pedagogical knowledge, 
and TPACK in the quantitative data also suggest that the student teachers developed some 
understanding of the complexities of teaching online. As a result, they are better equipped 
to overcome difficulties posed by new and different environments, tools, and applications in 
the future. This finding thus also highlights the acquisition of transferable digital-pedagogical 


competences through virtual exchange in initial teacher education. 
4.5.9. Proposals for developing digital-pedagogical competence 


The evaluation of both the quantitative and the qualitative data shows that overall the tools and 
applications used by the student teachers while engaging with the tasks/task sequences promoted 
in EVALUATE together with triggered reflections on their experiences, had a positive impact 
on their digital-pedagogical competence development. It has also confirmed previous findings 
(Fuchs, Hauck, & Muller-Hartmann, 2012) which highlight the fact that virtual exchange does, 
in fact, provide the ideal set-up for task-based digital competence development in initial teacher 
education. Recommendations for future competence development to this effect in the context of 


virtual exchange based on the insights gained from the project are as follows. 


It is necessary to raise awareness among future teachers that technology use in the classroom 
has benefits beyond new ways of presenting content to learners. It provides new opportunities to 
engage with materials and other learners, locally as well as across time zones and geographical 
distance, especially in the context for virtual exchanges which are — by default - mediated by 


technology and therefore require the use of online tools and environments. 


Sufficient time needs to be allocated in virtual exchanges for initial teacher education to explore 
the affordances of individual tools and applications (text only; text plus visual communication; 
text, visual communication, and audio; etc.): how they facilitate the learning process overall and 


how they allow the learners to reach milestones and — eventually — clearly defined learning goals 
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on their learning journey. As such, in the case of initial teacher education, these goals should 


include the acquisition of technological-pedagogical competence. 


Although it is clearly part of allocating sufficient time, the need for built-in reflection on 
technology use in the (future) classroom is paramount. Therefore, it must become an integral 
feature of initial teacher education offerings. As we were able to demonstrate, virtual exchange 


embedded into initial teacher education provides the ideal set-up to this effect. 


Virtual exchange provides a safe environment to experience challenges with regard to 
technology use in initial teacher education. Both challenges encountered and the strategies 
trialled to overcome these (including information about the nature of the challenge and the tool 
or application involved) should be systematically collected and made available in the shape of a 
repository for future student teachers — at least at institutional level, ideally regionally, or even 
nationally. Building and maintaining such a repository — even in its most basic form — requires 
commitment at institutional, regional, and/or national levels as well as funding for its creation 
and maintenance. Once in place, it will provide an invaluable resource for virtual exchange- 


based initial teacher education and beyond. 


4.6. Impact of virtual exchange on students’ foreign 
language competence: quantitative findings 


Foreign language competence development is complex and discontinuous, and takes place over 
long periods of time (Ellis, 2015, pp. 297, 307). Virtual exchanges are normally short term (in 
the case of EVALUATE, lasting approximately five to ten weeks). Without an intervention study, 
using corpus research methods, it is a challenge to demonstrate conclusively (e.g. by tracing 
‘uptake’ or other changes in output) that foreign language competence development has taken 
place. We have therefore relied, in addressing this question, on participant testimony. This 
takes two broad forms which combine quantitative with qualitative data. The first is derived 
from a survey of participants, administered following the exchange, the second is drawn from 


participants’ comments in an online reflective journal. 


Participants were asked ‘How (if at all) has your ability to use a foreign language developed 
in the course of the exchange?’ They were given the option of indicating whether they judged 
that their foreign language competence had (1) improved much, (2) improved a little, (3) not 
improved at all, or (4) actually got worse. It would be inappropriate to expect anything more 


than minor improvements in the course of a short-term exchange; “[mlJost people arrive at their 
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fluency only as a result of hard work, expended over a considerable period of time” (Crystal, 
2010, p. 388). Moreover, self-report data is necessarily subjective. Any attempt to calibrate the 
scale of any improvement would have been spurious. In presenting the survey results, therefore 
we have combined categories (1) and (2) to offer an overall indication of whether participants 
felt that their foreign language competence had been positively impacted by the experience of 


virtual exchanges. The results are as follows (see Table 3). 


Table 3. How (if at all) has your ability to use a foreign language developed in the course 


of the exchange?’ (combined data) 


elmo aes | 


Ability to interact with foreign language speakers 100% 


Ability to understand 100% 
Range of vocabulary 100% 
Accuracy of pronunciation 100% 


These outcomes are encouraging, since it appears that the aspects of foreign language 


competence in which virtual exchange participants most frequently report gains are the ability 
to interact with foreign language speakers and the ability to understand. These language skills 
are aspects of pragmatics, which deal with the (often indirect) ways in which meanings are 
made and interpreted in communicative contexts. They are clearly essential in interpersonal 
(and intercultural) encounters since they enable the growth of empathy and the development 
of an ability to convey it. In other words, they make possible the building of rapport. As for 
the other language skills, it is unsurprising that in a series of interactions which are focussed 
on the communication of messages and meanings (rather than on the learning of structures), 
a higher percentage of participants report lexical development (69%) than grammatical 


development 56%). 


Caution is required in basing findings on what participants say, rather than what they do. 
But EVALUATE learners’ insights into their language development are in line with recent 
research into the impact of technology use on foreign language learning. Dooly (2017) notes 


that “telecollaboration appears to be advancing the [...] paradigm towards what Littlewood 
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calls ‘communication-oriented language teaching” and explains that communication-oriented 
language teaching “emphasises the use of language in ways that are personally relevant to the 
learners while helping them develop communicatively, cognitively, and as a ‘global’ person 
through collaborative learning” (p. 125). Sykes (2017) argues that “telecollaboration via digitally- 
mediated tools is [...] an effective context for the learning of pragmatic behaviours as well as the 


application of patterns in authentic discourse” (p. 126). 


There is however, some cause for concern in the data. Fluency in speaking and accuracy of 
pronunciation in the foreign language appeared to benefit less from virtual exchange than other 


aspects of foreign language use. The reasons for this will be explored in a subsequent case study. 


4.7. Impact of virtual exchange on students’ foreign 
language competence: qualitative findings 


Perusal of the qualitative data provides an explanation of the scores reported above and enables 
us to draw a broader and more nuanced picture of foreign language competence development 
over time in EVALUATE. Applied linguists will be aware that not all the competences necessary 
for successful interaction were addressed in our survey. There are very straightforward reasons 
for this. The overall survey was long (it had 21 questions). Our questions needed to be succinct. 
There are some concepts which require explanation to be made clear to a lay audience. One such 
is pragmatics. Instead of asking about participants’ pragmatic competence, we asked instead 
about the ability to interact. The two are related but not coterminous. However, participants’ free 
text comments allow us more clearly to identify what appears unequivocally to be references 
to pragmatic competence. What they all have in common is that they show learners attending 
carefully to the signals they are receiving from their interlocutors, adjusting to perceived 
needs and limitations in a context of mutual learning, and using the foreign language to show 
understanding and appreciation of their partners’ efforts to communicate. This is pragmatics at 


work. 
4.7.1. | Pragmatic competence 


Pragmatics “considers language as an instrument of interaction, what people mean when 
they use language and how we communicate and understand each other”. It looks beyond 
the literal meaning of an utterance and focusses on how implied meanings are constructed, in 


the light of context. Pragmatic competence has two aspects. Sociopragmatic competence is the 


1 http://all-about-linguistics.group.shef.ac.uk/branches-of-linguistics/pragmatics/what-is-pragmatics/ 
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awareness of what forms of expression (known as ‘speech acts’) are appropriate in a particular 
communicative context. Pragmalinguistic competence is the ability to control one’s use of these 
linguistic forms. Here is what EVALUATE participants had to tell us about their development of 


pragmatic competence: 


“T think that I’ve developed my speaking and communicative skills during our video 
conferences and chats, where the main idea was not to show off about our language 


competences, but to be understood and efficient in our work”. 


“Maybe I have used more informal and simpler language than I am used to because at 
Uni we use academic language, but when talking with older people we use an informal 


register”. 


“As in every part of this exchange, I’ve practiced making my English understandable and 


adjusting my speech to my interlocutor’s level”. 


“However, what I believe I have improved is the way I explain thing[s] in English to people 
that have a lower level, I mean, I have learn[ed] to simplify my ideas in order to assure 


they are understood”. 


“TI learnt to use a lot of language elements concerning politeness, appreciating and 
compliments that I usually don’t use in this amount. But it was important to show and 
articulate that you appreciate the others’ effort and work because they wouldn’t know it if 


you don’t tell them directly or text them”. 


The learning recorded by participants in these comments is not that of new items, or rules, 
of language. In fact several comments make it clear that participants were interacting with 
partners whose level of proficiency in a shared lingua franca was inferior to their own. What they 
learned, however, were advanced communication skills: the use of simplified lexis and syntax 
for purposes of clarity; the adoption of a less formal register to adjust to their interlocutors; 
and above all, sociopragmatic awareness (‘show ... that you appreciate the others’ effort’) and 


pragmalinguistic control (‘I learnt to use a lot of language elements concerning politeness’). 


Pragmatic competence is indispensable for successful communication. Without it, little genuine 
understanding, let alone agreement, will be reached. It has a “critically important role [in] 


digital contexts” (Sykes, 2017, p. 127). Yet it is difficult to teach; in the words of one expert, 
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“the challenge for foreign or second language teaching is whether we can arrange learning 
opportunities in such a way that they benefit the development of pragmatic competence in L2” 
(Kasper, 1997, n.p.). Virtual exchange, based on appropriately designed tasks, is clearly rich in 


such opportunities. 


4.7.2. Lexical competence 


There is a close link between pragmatic and lexical competence development, as the acquisition 
of additional lexical formulas equips learners to perform a wider range of pragmatic functions. 
Both of the following testimonies show virtual exchange participants applying themselves 
consciously to learning vocabulary, in the shape of useful formulaic expressions. The second 
suggests clearly that the learning of such functional formulas plays a role in developing 
pragmatic competence in the foreign language, as argued by Kecskes (2014). Here, the emphasis 


is on friendly, informal greetings: 


“We tried to use different types of expressions and verb forms to practise vocabulary and 


make sure that writing messages did not become monotonous”. 


“Yes, we learned many new elements like typical friendly greetings. We only knew informal 
greetings and we didn’t know our virtual exchanges partners. I used words like ‘greetings’, 
‘hugs’, ‘first of all’ etc. I jotted down some of the informal ones after I had learned a bit 


more. This is a useful way of learning more of these kinds of words”. 


4.7.3. Grammatical competence 


Though fewer in number than comments recording the development of pragmatic competence, a 
number of free text journal entries nonetheless make clear that the development of grammatical 
competence was also a feature of the exchange. It appears to have been engaged in consciously. 
We reproduce here comments making reference to specific aspects of grammar as being more 
convincing than broad generic statements. They are of two kinds. Firstly participants reflect 
on their own language use (‘more modal verbs than I use to use’ [sic]). They also recount how, 
as trainee teachers, they honed their pedagogic techniques and skills (e.g. illustrating and 
explaining the use of the present and future tenses), or summoned up the confidence required to 
correct a partner’s grammatical errors. Teachers of foreign languages will understand just how 
important the acquisition of such confidence is. It is significant that virtual exchanges afforded 


participants a space in which they felt able to develop in this way. 
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“The elements of language that were relevant to our task that we designed through the task 
assigned to us were as follows: descriptive language (adjectives), present tense (describing 
culture), new vocabulary related to the celebrations of New Years, oral presentation, and 


past tense (writing a reflection of how the lesson went)”. 


“Probably more modal verbs than use to use, mostly to make suggestions or requirements”. 


“In this task, I had to review the future simple because one of our virtual partners made 
a mistake. He omitted the main verb when performing a present simple sentence and 
I tried to explain to him his mistake so that he did not make it again. Also, I incorporated 
some examples so that it was easier to him to understand the explanation and so that he 
could see other different contexts. I managed to use funny sentences because I think that 
it can help him to memorise the grammatical theory. I hope that the explanation was clear 
enough for him and that it can help him in future times to communicate better with other 


people”. 


“Gendered nouns (agua,encantado)”. 


“Confidence to correct grammatical mistakes”. 


4.74. Confidence 


Though they were not specifically asked this question, a number of participants indicated that 
their confidence in using or speaking a foreign language had improved in the course of the 


exchange. Even those who did not specifically use the term, wrote of: 


“Feeling able to express myself”. 
“Audacity in using English in real-world situations”. 
“Ability to express your ideas with a foreign language”. 


“Ability of giving your opinion in the other language”. 


Confidence is extremely important to foreign language learners, precisely because it empowers 
them to produce more of the target language, taking risks when necessary and learning in 
the process. Gains in confidence are regularly reported by participants in telecollaborative 
exchanges. It may be that this again is linked to a focus on the communication of meaning, 


rather than on formal accuracy. This is a hypothesis that requires further research. 
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4.8. Concluding comments 


On the whole, the data suggests that virtual exchange has a positive impact on foreign language 
competence development. Only in relation to two aspects of foreign language competence, 
‘fluency in speaking’ and ‘accuracy of pronunciation’, was the percentage of participants 
who reported some development outweighed by those who indicated that they had made no 
improvement. The reason for this is relatively clear. As one respondent indicated: “[t]he only skill 
that hasn’t improved is speaking fluently, because we had no opportunity to speak within our 


project work”. That is a message for future organisers of virtual exchanges for whom improved 


foreign language competence is an intended learning objective. 
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5. The experiences of teacher trainers 
using virtual exchange 


5.1. Studying the role of teacher trainers 


As seen in the previous sections of this study, the main research questions asked in the European 
policy experiment have been related to the impact of virtual exchange on student-teachers’ 
competence development. However, demonstrating that virtual exchange has a positive 
impact on students’ foreign language, intercultural, and digital-pedagogical competences is not 
sufficient to ensure that this learning approach will be implemented on a large scale in European 
institutions of initial teacher education. It is also important to understand the perspective of the 
teacher trainers who took part in the EVALUATE exchanges and to delve in-depth into their 
experiences of running virtual exchanges as part of their courses in initial teacher education 


and within the context of teacher education in their countries. 


In order to achieve this, the research team contacted teacher trainers from the two rounds of 
projects, inviting them to take part in either written (interview questions answered through 
email) or online and/or face-to-face interviews. Interviews with a total of 31 different teachers 
were collected. The aural interviews were then transcribed and uploaded into the NVivo data 
analysis platform. The profile of the teachers who took part in the study reflected the overall 
profile of the teachers participating in the EVALUATE exchanges: they were teacher trainers 
mainly from the project’s ‘partner’ regions and countries — Spain, Portugal, Hungary, and Baden 
Wiurttemberg, Germany. The large majority (29/31) were also novice telecollaborators — this 
was for most the first time they had organised a virtual exchange in their classes. This was a 
deliberate choice on our behalf as we wished to study the impact of virtual exchange on teachers 


who were new or relatively new to this activity. 


The two researchers involved in this part of the analysis then individually coded and recoded 
the data (Chilisa & Preece, 2005). This was a two-step process wherein the data were first 
thematically coded according to the main points that were detailed by the participants and then 
a second coding was undertaken by the same researcher after a week to ensure that the first 
emergent themes were reliable. Next, the finalised thematic codes were exchanged between the 
researchers, corroborated, and the main themes identified and correlated to the initial research 


questions listed here: 
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AYVAakchenqicctsmualsm bay oy-Cemoy ma vvabapval:w-Mrsbuaur-UM->.colar-valex-moyamealcmucrcloabuatca-balemoyame als 


professional practices of the teacher trainers? 


AYi"Aatchemmed at-U0=yabexcrcuur- 0 alol(oyemyo)yuo) o)(-yankmmeDCoMmdalyiamcyaleolobalccyamn\savl(-meuebabapbalsama als 


exchanges? 


Based on their experiences, what factors do the teacher trainers consider 


important for the success of virtual exchange? 


What were the reactions of their faculties and institutions to the introduction 


of virtual exchange? 


AYVAatchapmeXcoyaanantsyaXotsuuloyatsmoomuatsmucts\e aX=) muna bha=)ucwatshic-m coven sxonoler-luloyarey Molsveitsakeyal 


makers and their ministries of education in relation to virtual exchange? 


5.2. The impact of virtual exchange on 
their teaching and professional practice 


Many teachers remarked on the profound impact their experience had on their own teaching 


practice and approaches. The highlighted points were: the teachers were very satisfied and found 


the experiences to be generally positive for both themselves and their students, underscoring 


gains such as the broadening of their pedagogical knowledge (new ideas, abilities, and confidence 


to implement further telecollaborative exchanges) and other unexpected opportunities 


including new (physical) mobility agreements and student visits. These findings are illustrated in 


Figure 26. Some of the teachers also mentioned that taking part in these projects compelled them 


Generally Very 
positive satisfied 
experience 


: Other 
Ua ta iecet Impact ((Uat-).4 el-(ak-10)) 
now-how opportunities 


New Expanding 
fex=ter-Tefeye) (er=1 | mobility 
ideas opportunities 


Figure 26. Main outcomes teacher impact 


to be more innovative in the face- 
to-face classrooms, bringing into 
play more participative teaching 


strategies. 


Of the points mentioned, one of 
the most highly referenced (and 
unexpected) points that emerged 
from the interviews was the 
development of new professional 


or collaborative opportunities and 
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expansion into other virtual exchange practices based on similar patterns of practice. Several of 
the teachers explained in detail how new possibilities for collaboration arose from the virtual 
exchanges. These ranged from Erasmus agreements (physical staff exchange), to collaboration 
on publications and presentations in academic conferences regarding their own experiences. 
One teacher mentioned that her university had decided to continue with the initiative as a 
sustained practice (note that in the following quotes, the teacher’s names and other potentially 


identifying markers have been removed to retain anonymity of the participants): 


“It’s something that has served as a basis this year were [sic] doing a similar experiment I 
suppose it’s what you’d call it we’re having a similar experience with a Danish university 


with students in a different subject that ’m sharing with another colleague”. 
There was also mention of ‘non-institutional’ collaboration: 


“We haven’t discussed an Erasmus mobility but I would like to invite the two teachers 
from Holland to come and teach here in my faculty and organise a workshop for the other 
teachers in the faculties — and invite students to come and talk about their experiences. 
Because I think this would be suitable not only for students of languages but also [for] 


students of other faculties”. 


This type of unexpected collaboration and physical mobility was not only in relation to 


themselves, the teachers also noted that the students had arranged for ‘impromptu visits’: 


“The [partner class 1] group is now with intention to try to carry out a ‘study visit’ to [the 
partner institution] next year. Teachers are also considering joint projects around this 


theme [of virtual exchanges]”. 


5.3. The challenges and problems 
which teacher trainers encountered 


As mentioned above, the teachers were generally very positive about their experiences, citing 
both student and professional gains as reasons to carry out virtual exchange. At the same time, 
the teachers were quite honest about the problems and challenges they encountered during the 
process. Of course, virtual exchange is a highly complex practice and the challenges that arise are 
complex and multi-layered, although we attempt to ‘tease out’ the main points emerging from 


the interviews in this section. The most frequently mentioned issues were (in order of most oft- 
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cited): technology, student resistance, time management, task design, intercultural issues, and 
difficulties arising from the teacher partnership. These themes are illustrated in Figure 27. Due 
to the qualitative nature of the interview data, we do not consider exact numbers of referencing 
as the only indicator of frequency 
because one teacher may mention 


Student 


Yel alate) Coley] ; 
resistance the same topic several times 


in the same interview (if it is a 
Time 


management Challenges topic of key interest for them, for 


instance). Thus, recurrence was 
Intercultural Se considered as a sum factor of (1) 
ae approaches being mentioned by the majority 
of the teachers and (2) mentioned 
Figure 27. Main outcomes: teacher challenges more than once by more than one 
teacher in the interviews. 


5.3.1. Technology 


Difficulties arising from the technology were usually either related to differences in access and/ 
or knowledge of the technologies between the two groups or lack of diversity in the available 


technological resources: 


“My students were not able to form strong relationships due to the weak internet 


connection in Italy and having limited time to interact online”. 


Inevitably, the matter of technology is directly related to issues that arise out of scheduling 
overlaps and time zones (another common problem that was brought out in the interviews). 
These points are important, particularly in relation to synchronous online meetings and the 


timely exchange of geographically-distributed student activities: 


“The day we wanted to do the skype conversation we asked for the salon de actos 
[translation: Conference Hall] and the technology did not work there. Finally I used my 
phone and it was not good at all. So if technologies are related in your work, that’s not easy 


for us”. 


More than one teacher mentioned that they were surprised by the lack of ‘technological 


awareness’ of their students, despite being the generation considered to be ‘digital natives’. At 
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the same time, the teachers also mentioned the resourcefulness of their students for resolving 


communication difficulties: 


“So we divided the students into groups and we gave them email addresses. They took off 
and collaborated... everywhere. And for me, I’m a total control freak. [laughs] Google Docs, 
Facebook Messenger... they invited us into everything so we could see what was going 
on. And I was quite surprised because their collaboration was quite dynamic actually, 
choosing whatever they wanted. But I don’t think I would do that again because it was a 


bit hard to control. I think you need a base camp”. 


5.3.2. Student resistance 


Despite overall positive feelings about the experience, the teachers did notice that there was 
some ‘pushback’ from the students themselves although they felt that in the end the students 


demonstrated learning and intercultural gains: 


“Students complained a lot in the process and said it’s not possible, we can’t do that and 
we have to be in the same room and no... but in the end, I think they were okay with it and 


the products were nice”. 


The teachers provided different possible reasons for student resistance, explaining that it is a 
‘normal’ part of any pedagogical experience; that the pressures of completing a series of tasks 
during a specific window of time affected students’ perceptions and different expectations 


regarding the exchange. 


5.3.3. Time management 


As it has already been mentioned above, time management is woven through other difficulties 
referenced in the interviews. This issue is related to both short periods of calendar overlap 
(scheduling) and time pressures of completing the tasks on schedule in order to fulfill their 


partners’ requirements: 


“The main problem was the issue of time. Because we were paired with the Swedish 
university and the academic timetable is very different to ours so the overlap between 
the subject being taught in Sweden and the subject being taught here was quite a narrow 


window so we felt the whole time we were under a lot of pressure to get the students 
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through the different stage or the different the [sic] tasks —- in fact we had to reduce the 


tasks”. 


5.3.4. Task design 


As with time management, the issue of task design is not as simple as the category might 
suggest. Problems with task design range from different teacher interpretations of the task 
implementation and outcome to difficulties in time management of the task, to the amount of 


task monitoring by the partner teachers: 


“Tt is fundamental for teachers to organize and monitor; interference is not good. Students’ 


conversations were personal and social and should be allowed to develop that way”. 


5.3.5. Intercultural issues and difficulties arising from the teacher partnership 


Intercultural issues emerged from both student and teacher perspectives. In reference to the 
students, intercultural questions were mostly related to different approaches to the assigned 


work in the virtual exchange: 


“They had lots of complaints about their partners in [the partner class]. I don’t know if it’s 
been a coincidence, I don’t know if it’s just that it happens in all groups, that some students 
are responsible, they’re hardworking, they get their work done, but some others are not. 
So the thing is that they didn’t, some of their partners did not reply on time, they didn’t do 


the work; they weren’t willing to do video conferencing”. 


In all of the interviews, the teachers balanced their negative comments with affirmative 
statements of the benefits they gained from taking part in the exchange. Several noted that 


negative feelings that arose were not pervasive throughout, nor were they always long-term: 


“There was a phase when we both had the feeling that, it felt like our students were 
blaming it on [their partners] and [their partners] were blaming it on our students. ‘They 
didn’t do anything’, or they said ‘They don’t do anything so we can’t continue’ and so on. 


Maybe it is also normal that they wait till the last minute and then they do it”. 


In at least one exchange, the teachers noted that with sufficient support the participants in the 


exchange were able to discuss and work out and negotiate their differences: 
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“Because [my class] had an idea for the design of the final project. So they came up with 
an idea and presented it to the [the partner class]. So the [the partner class] just changed 
it. They thought that it was what they were supposed to do. But [my class] were completely 
upset. They thought [the partner class] would praise the product and just comment on 
it. So they were completely upset and both groups complained to their lecturers. So my 
partner and I got to know about it. So I talked about it to my partner and said we should 
leave it to the students. We told them — you are capable of solving the problem now. This 
is a cultural problem and this is the type of intercultural situation we were talking about 
and will you be able to manage that? And they did. They talked about it. And that was the 
thing, they didn’t stop when they were upset. They kept on talking about it. And of course 
we had to push them a little bit. And they were able to solve it”. 


Different work approaches were also evident between the teacher partnerships, contributing to 


some communication gaps: 


“Some students from her class and from my class left the project and after that the 
communication between the two teachers was less fluid. She said in an email ‘talk to your 
students, find out why they are not working’. I replied that it was the problem of the two 


groups. After that the project went better but we never exchanged any more emails”. 


5.3.6. What is the impact of virtual exchange on their students? 


The interview data reveals a wide range of positive learning outcomes which the teacher 


trainers feel that the virtual exchanges had on their students. These outcomes are summarised 


Motivation 


Tate lUr-let— Tate Uretei 
practice exposure 


Intercultural 


Impact on students 
awareness 


(Cross-disciplinary) 
Taal tate merlin 


Digital gains 


Figure 28. Main outcomes of student impact 
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in Figure 28. In general, teachers found that virtual exchange enabled opportunities for learning 
which were not normally available to students in the traditional classroom. For example, teachers 
reported frequently that students had found the exchanges motivating as a learning activity and 
that it had allowed them to learn and practise their English in an authentic way with speakers of 


other languages — something which is not easily achieved in monolingual classrooms: 


“For my students this has been very motivating. They told me that they had enjoyed being 
in contact with another group, to know other cultures, to talk about their subject with 
others....this is a direct contact for them. Normally, they wouldn’t be able to practise their 
English. Even though this was with non-native speaker speakers [sic]. But I suppose that 


makes sense too because we are always talking about ‘international English”. 


They also mentioned that students had developed both greater awareness of different cultural 
perspectives as well as learning information about their partners’ cultures and the different 


systems of education. One practitioner explained: 


“At the end of the exchange the reaction was very positive because the students said they 
had learned a lot from their partners and that was, for me, very satisfying. Because I 
realised that the students had understood the essence of what we wanted to develop in 
this project —- which is put yourself in the position of the other, understand them and, many 
times, give way. They also told me that they had learned the contents of the course better 


because their international partners had helped them see the contents from another point 


of view. And they learned how other people work”. 
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As the project had been based to a great extent on issues related to education, other teachers 
found that virtual exchange also enabled students to learn about differences between educational 


systems: 


“However, I think it went well, as students got to know the education system in Brazil and 
discussed common themes in both education systems, such as, for example, lack of success 
or assessment. In terms of the education of future primary teachers, this experiment was 


enriching and gratifying for students”. 


Finally, teachers spoke regularly about how students had developed their digital competences as 
well as their understanding of how to use technology in a pedagogically efficient and innovative 
way thanks to the exchange. For example, one teacher mentioned that “[mJany of them didn’t 
know the tools that we used. Like Padlet, voicethread we used, well nearly all of them they 
didn’t know any of them and they found them useful”, while another practitioner explained 
how students had explored how they themselves could integrate virtual exchanges into their 
own Classes: “I think they learned how to set up such an exchange with their students, we also 
reflected a little bit every time on what kind of support you could give your students if you do it 


in for example secondary schools”. 


When asked about students’ use of online tools during the virtual exchanges, practically all the 
teacher trainers reported similar experiences. The EVALUATE team had originally prepared 
a Moodle platform (a well-known and easy to use virtual learning environment) where each 
exchange was assigned a course for their online interactions and digital products. However, a 
large majority of the teachers reported that students had been quick to move their interactions 
away from the Moodle forums and on to more interactive and personal social networks such 
as WhatsApp, Skype, or Google Docs. A representative example of how students worked is 


explained by one of the teacher trainers: 


“From the technological point of view all students created a gmail account, registered on 
the Moodle platform, used Google Docs to share information, VivaVideo to design short 
introductory personal and group videos addressed to their colleagues in Brazil, used 
PowerPoint and Prezi to present final project work, and also communicated via social 
media, such as Facebook, Instagram and WhatsApp. For synchronous communication 
during several consecutive weeks, the teachers in charge of the project in both countries 
used Skype and Zoom. When students communicated synchronously [outside of class], 


twice, the Zoom platform was used”. 
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It appears that while using a virtual learning environment such as Moodle may be more practical 
for teachers as they try to follow their students’ interactions, students prefer to use online 
communication tools which are considered to reflect their daily online communicative practices 


and also allow for instant feedback. One teacher trainer from Germany reported: 


“One student said to me: ‘Once I used the WhatsApp chat it was the first time it felt like real 


people, before that it felt like texting to a computer””. 
Another teacher explained: 


“What they wanted was instant synchronous communication. They wanted instant replies. 
I don’t know, they could see the pictures of the people and their names. They said that 
Moodle looks old fashioned. It’s easier to connect via Facebook and write via Messenger 


because they can look at the photos and learn more about the person”. 


As a result of this approach, the Moodle platform was used almost exclusively for submitting 
finalised documents and project outcomes to the teachers. Teachers seemed to be generally 
comfortable with this approach although a small number of informants did suggest that the 
mix of tools led at times to confusion and that it had resulted in a loss of control by the teachers 
as they did not always have access to the myriad of collaborative documents and text and oral 
conversations which were taking place. And while this ‘loss of control’ of the technology being 
used is ostensibly a positive aspect of the learning process (gradual autonomy and learner 
control), the remark seems to indicate a feeling by the teacher that she no longer knew what was 
occuring in the exchange nor had full access to students’ language use within the parameters 
of the class. We choose not to evaluate this statement from a pedagogical perspective, but 
rather we focus on the need for the practitioner to feel comfortable with the virtual exchanges 
and therefore include it as an indicator of a challenge for the teacher regarding the Moodle 


platform: 


“TJ don’t think I would do that again [allow for a mix of online communication tools] 
because it was a bit hard to control. I think you need a base camp. ...And I think my 
partner realised that too that you do need this base camp. You can send them off into these 


different satellites but they need to come back home and report”. 


These comments most clearly reveal a possible mismatch between the digital practices of the 


teachers and their students. Previous studies show that educators often feel that they face a 
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“constant challenge of refining teaching and learning techniques to keep up with the increasing 
demands and expectations [regarding technology]” (Duncan-Howell, 2012, p. 827). Often times 
learners are what might be called ‘digitally expectant’, that is the assumption that technology 
will be integrated into teaching practices nowadays and the teachers feel they are not tech- 
savvy enough to meet these expectations. These issues appear to have come to a fore in these 
exchanges as well. Added to this, the nature of telecollaborative exchanges implies effective use 


of communication technology which can be interpreted as additional pressure. 


5.4. What do the teacher trainers consider 
important for successful virtual exchange? 


The qualitative analysis of the teacher trainers’ interviews allowed us to identify a series of 
good practices in virtual exchanges which practitioners considered important for the success 
of this type of learning (see Figure 29). The first two of these good practices are related to the 
teacher trainers themselves and how they approached their working relationships between 
their partner teachers, while the remaining three are related to how the virtual exchanges are 


structured and carried out. 


Joint Regular 
collaborative Cofoyaatantelalcersia(eyal 


tasks between partners 


Full Face-to-face ETalaliare| 
integration in Reasons for success meeting together 
curriculum oX=3 Ke) a=) ar-lale| face-to-face 


Use of Institutional 
videoconferencing support 


Figure 29. Reasons for success 


5.4.1. Virtual exchanges work better when teachers have an opportunity 


to meet their partner and plan together beforehand 


Throughout the interviews with the teacher trainers, it became evident that those who considered 
their exchanges to have been a success related this directly to a good working relationship 


with their partner teacher. This type of relationship was achieved by, first, actually having the 
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opportunity to get to know the partner in a face-to-face context before the exchange and, second, 
by maintaining constant contact via email or videoconferencing with the partner teacher during 


the exchange itself. 


Many of the teachers had had the opportunity to meet their partners during the workshops 
which were held in Padova, Italy, and Leon, Spain before each round of virtual exchange. For 
many of those teachers, this was an opportunity not only to plan the exchange together but 
also to establish trust and to develop a working relationship together. One Spanish teacher 
commented “[oJur relationship was strengthened by the two days we spent working together in 
the Leén workshop” while another recalled “[m]y partner and I worked together in the training 
workshop in Padova and we really connected well together. We worked a lot because we wanted 


to get a lot done together”. 


For some of those who did not have the opportunity to meet together physically, the option of 
videoconferencing sometimes appears to have been sufficient. However, in all cases, it appears 
that this ‘getting to know you’ phase must go further than merely agreeing on the practicalities 
of the exchange and needs also to involve the development of a relationship of trust between the 


teachers. For example, a teacher explained: 


“She [my partner] hadn’t been at any of the workshops. So I was wondering how was this 
going to go. But she was very helpful and we skyped a couple of times. And I think that’s 
very valuable because when you get to actually see each other and actually talk... and the 


time difference was that I was at home in the evenings so it became more casual”. 


When exchanges were not reported as being successful by the teacher trainers, the lack of initial 


contact between the instructors is mentioned as one of the negative factors: 


“Tt is more difficult to carry an online exchange if you do not have a face-to-face meeting 
with the partner institution’s leader. It is important to know beforehand what each other’s 


expectations are”. 


5.4.2. The relationship between the partner teachers 


requires close attention and flexibility 


When the virtual exchange begins, the relationship between the two teachers and their ability 


to communicate and collaborate effectively together also becomes key to whether or not 
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teachers consider their exchanges as being satisfactory. This involves, first of all, regular contact 


throughout the exchanges as this teacher highlights: 


“The reason why it works so well is definitely the relationship between [his partner] and 
me. For this telecollaboration we spent an awful lot of time together. We basically mailed 
at least once a day and we are very close as regards the preparations and everything. We 


got to know each other very well. It’s very personal relationship and that helps a lot”. 


Other informants made clear that this teacher-teacher collaboration must, again, go further 
than simply exchanging information and trying to fit the exchange into pre-existing curricula, 
but must also involve an active interest in the partner teachers’ context and a willingness to 
adapt and be flexible when necessary. One informant explained: “[t]he relationship with my 
partner was of mutual respect, of collaborative work, and of great disposition on the part of the 
teachers” while another reported “we were able to adapt to each other’s plans and to appreciate 
each vision and suggestion and we tried to really coordinate our syllabus. I think that we made 
a very positive work in adapting to each other’s syllabus and to create [sic] a new one where we 


could accommodate each other’s perspective”. 


The term ‘flexibility’ appears repeatedly in the interview data as being one of the keys to a 
successful working relationship for teachers running virtual exchanges together. A practitioner 


from one of the exchanges reflected: “I am surprised to realise once again that there is a need 


to organize every bit in detail and to be ready to fix unexpected problems. And to be flexible!”, 
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while another teacher from Holland concluded “[iJt’s actually not that hard to carry out sucha 


project ifyou and your partner really want it. You just need to be a little flexible and motivated”. 


When instructors did not maintain fluid contact with their partners during the exchange, this was 
commonly mentioned as one of their regrets or one of the keys to a lack of success in exchanges. 
When asked what advice they would give other teachers planning to take part in virtual exchange, 
one practitioner suggested the following: “[c]ommunicate with your partner face to face once every 
week or so— more than I did”. Furthermore, when two instructors who reported disagreements or 
annoyance with their partners, this was clearly based on a lack of regular communication together. 
For instance, this first example shows a case where lack of coordination between teachers led one 


of them to become frustrated with the lack of balance in workload: 


Interviewer: “Did you have any kind of workload distribution? In terms of organizing the 


exchange?”. 


Teacher trainer: “(laughing) Well, (aughing) I did all the Moodle things and then I asked 
her all the time and she always said ,Yeah yeah that’s fine, that’s fine! So we didn’t really 
develop this together...”. 


This next example also describes the consequences of the lack of communication between 
the partner teachers during a class where students were involved in synchronous discussions 


together: 


“So there were two or three groups that worked really really well and there was one 
group that was just sitting there [in class] and didn’t get any replies from their [partner] 
group which was also sitting in class. And then I couldn’t communicate with [the partner 
teacher] because she didn’t reply to her emails and she didn’t have any other social media 
because she didn’t have a smartphone. So I had to tell one of my students to tell her partner 
to tell [the partner teacher] that one of the groups wasn’t working. So that made it very 


complicated”. 


5.4.3. Tasks with joint collaborative outcomes are the most effective 


for learning — but are also the most challenging for students 


When teachers spoke about what were the keys to successful learning outcomes for their students, 


many of them referred to the benefits of virtual exchange tasks which involved requiring 
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students to collaborate closely together in order to produce a joint product or to complete a 
project. Teachers found that it was during these intense periods of ‘high stakes’ negotiations and 
collaboration that students learned most about cultural differences and about the challenges of 


online collaboration. One practitioner reflected this widely-held belief in the following: 


“Producing something together is the most important part of the whole project. It’s a hands 
on activity and they have to generate something and this is where they realise cultural 


differences, how people go about things and how they deal with it”. 


However, getting students to collaborate together in close coordination also brings with it a 
series of challenges. Problems with partner response times, timetabling differences, and bad 
internet connections became all the more visible when students were under pressure to complete 
projects which depended on input from both classes. This led, at times, to students blaming each 


other or feeling let down by their partners. An exchange practitioner explained: 


“at the end they really worked quite hard to finish their products... But there was a phase 
when we both had the feeling that..., it felt like our students were blaming it on the partners 
and the partners were blaming it on our students. ‘They didn’t do anything’, or they said 


‘They don’t do anything so we can’t continue’ and so on”. 


This is definitely a challenge for virtual exchange - the best tasks are often the most complex 
while at the same time, the need for the exchanges to be meaningful and with a purposeful 
task was highlighted by the teachers. This implies the need for careful planning, including 
consideration of the time needed for the activities and sub-tasks that the students are expected 


to complete. 


5.4.4, Virtual exchanges work best when they are integrated 


into classes and students receive credit for their work 


Teachers also reported that another key to success was the degree to which the virtual exchange 
could be integrated into courses. This meant that exchanges needed to be related to course content, 
worked on during class time, and be credit-bearing activities. One practitioner highlighted this 


strong approach to virtual exchange integration: 


“The exchange was integrated into both partners’ course classes. The chosen theme was 


according to the objectives of our two courses. In all classes, a period of time was always 
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reserved to ask questions about the tasks, to comment on the exchanges, and for the 


students to carry out their interactions with their partners”. 


Those teachers that did integrate the exchange believed that this was one of the keys to successful 
learning. For example, one teacher explained: “I would highly encourage it and suggest that they 
weave it into their course so it is a frequent topic of discussion”. Another teacher concluded 
that acting as an intercultural mediator for her students during the exchange had had a 


transformative impact on her students’ learning: 


“We have acted as mediators in the exchange and that was a very satisfactory task. 
Speaking from [sic] myself, I could see how my students changed their perspectives when 
sharing with the class their concerns about the communication with their partners. And 


I think the role of the teacher is very important here”. 


Similarly, those who did not integrate the virtual exchanges fully into their class programme (not 
allowing time in class or dealing with the exchange as supplementary activities) often reported 
regretting this. One teacher reported this as an important realisation after the fact: “I learned 
that... I should have allocate[d] class time to do exchange projects. I regretfully did not integrate 
the exchange into my classes — only for clarifying assignments”. Another also attributed a lack 
of credit for the exchanges as one of the reasons why students had stopped participating in the 
exchange: “[i]t wasn’t obligatory for [her partner’s] students and as it got more complex students 
started dropping out. Thirteen fell to seven. Her incentive was for them to do two essays instead 


of three! Very little!”. 


5.4.5. Virtual exchanges work best when students use 


videoconferencing/synchronous communication 


A large number of teachers underlined the use of videoconferencing between the partner classes 
as being another important factor for the success of exchanges. Teachers used videoconferencing 
technology in different ways. Some required their students to work in small groups using 
desktop videoconferencing tools such as Skype or Zoom, while others organised ‘class to class’ 
videoconferencing at key stages of their exchanges such as in the first or final weeks of their 
projects. The teachers reported that using videoconferencing meant that, in the words of one 
practitioner, “they saw each other as ‘flesh and bone people” and contributed to giving the 
students a sense of who their partners were. One teacher explained in her interview: “[i]n the 


past collaborations we only used [a text-based messaging app]. But that’s not enough. They really 
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need to see each other”. This explanation from a different quote captures how many teachers 


understood the difference between videoconferencing and text-based communication: 


“What I noticed was that the students would have really appreciated a live, oral 
communication between the students. Because everything was asynchronous. And that 
was a problem. Because when you read a text, it sounds cold. But when you hear someone 
saying something, it creates a more personal connection. So one of the things which we 
missed is that there were no personal relationships created. It was more like a work 


agreement and there was a lack of trust”. 


Of course, videoconferencing also brings with it certain challenges and problems which text- 


based communication does not have. There are practical challenges such as finding a time 


when both classes can be online together. There are also risks about how the interaction is 
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structured during the conference. For example, one colleague reported the following experience 


of videoconferencing with a partner class: 


“Then, in the final session, [my partner teacher] tried to get as much out of the conference 
as possible and because her students had like hundreds of questions and my students 
didn’t have time to ask any questions. And also they didn’t realise that the camera was 
on when we had this skype conference so my students could see her students using their 


smartphones and one student was reading in a book”. 


Finally, there is the pedagogical challenge of achieving in depth discussions on a class-to-class 
level in a live videoconference. The following teacher suggests that videoconferences need to be 


carefully integrated with text-based communication in order to be fully effective: 


“But, was [videoconferencing] helpful for learning the syllabus? I think it was an important 
social and cross cultural encounter, but not a learning encountering [sic] as it is meant to be. 
The resources and facilities didn’t facilitate a fruitful exchange — it was difficult to involve 
all the students in a reflexive dialogue and to make them carefully analyse each other’s 
contributions and to reflect and comment on it. Next time, we might need to have written 


contributions first, analyse it and then discuss and comment on each other’s work”. 


5.5. Reaction of faculties and institutions 
to virtual exchange 


When teacher trainers spoke about the role played by their institutions in the virtual exchange, 
there was a mixed set of responses. Various practitioners reported that their heads of department 
had expressed interest in the exchange and others mentioned how the projects had been reviewed 
and recognised by their faculties’ scientific committees. One colleague spoke about how she had 
been awarded a teaching excellence grant thanks to her work on the project. Others mentioned 
how their departments anticipated that their experiences using virtual exchange would lead to 


more teachers taking up this activity: 


“They have been both interested and supportive. I have been asked to give a short report 


on the project at a department meeting”. 


However, many others felt less institutional support. Some were critical of the lack of technical 


support and facilities which their institutions provided. For example, one teacher complained 
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“(like for example the infrastructure is not very supportive, for example, we don’t have reliable 
Wi-Fi, the computer room is terrible” while another explained “the videoconference facilities 
and resources didn’t work out. Next time, I will also know better what I will need and I will make 
my needs more clear to our services”. This teacher offers helpful insight for creating a better 
understanding between teachers and their institutions as regards previewing and ensuring that 


technology requirements are in place prior to the virtual exchanges. 


An interesting reflection that emerged from the teacher interviews was the fact that these 
virtual exchanges were not carried out by isolated ‘innovators’ but rather within the context of 
a large-scale Erasmus+ initiative. Doing so seems to have encouraged department and faculty 
decision makers and technical staff to pay more attention to the activity. One teacher described 


her institution’s reaction this way: 


“Well, in my building, for example, the people in charge... they were very kind, which is 
very strange [laughing] mostly because I sold the [virtual exchange project]... ‘we have 
this Erasmus with the [partner country], we don’t want to be just like... they gave me a 
good class, we had good conditions, things were working, and they were more or less 


pretty good. They are not normally but they were very good”. 


Similarly, in a different interview, one teacher explained the importance of various members of 


her institution taking part in the EVALUATE programme: 


“J tried to get them to bring the press to our faculty years ago to show them how we were 
doing synchronous communication but they weren’t interested. But now, because there 
are more of us, they are rehabilitating the language lab so we can do telecollaboration 
in it. But we need more people. For these things to work, you need lots of people doing it. 
Until you have more people doing something, you won’t get anywhere. You need union. 
That’s why I think it’s so important to spread the news in the university, so others will start 


doing it too”. 


These findings would suggest that in order for virtual exchange to take root in departments of 
initial teacher education, it is not sufficient for isolated innovators to begin using the activity in 
their classes. Instead, the experiences of the practitioners underscore the notion that decision- 
makers will pay more attention to these activities when they are carried out within the context 
of external programmes such as EVALUATE or Erasmus? Virtual Exchange and when more than 


one teacher at the institution is involved in the initiative. 
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5.6. What recommendations do the teacher trainers 
have for their institutions and their regional 
and national ministries? 


When teacher educators were asked what steps could be taken to promote virtual exchange more 
as a pedagogical tool in initial teacher education, some interesting proposals were made (these 
are summarised in Figure 30). First, a large number of practitioners identified the importance of 
awarding academic recognition for teachers who undertake virtual exchanges in their classes. 
The respondents constantly referred to the time which the organisation of virtual exchanges 
requires and they called for their institutions to award recognition of this extra work in the 


form, for example, of a reduction in teaching hours: 


“[We need]... a recognition of what we are doing. I believe teachers get motivated when 


what they do is valued. And this can be valued by giving them recognition for their work”. 


Another teacher suggested that virtual exchange be included among the many academic 


activities which lead to a reduction in teaching time: 


“that’s what they are doing with everything nowadays, with all the other extra things that 
we do, ...including it as part of the teaching load, even if it’s just very little, like half a 


credit. And that might encourage some other teachers, because it is time consuming”. 


However, teachers also warned that virtual exchange should not be made obligatory at 
universities and that teachers 
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Recognition Reduction 
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teacher training and changing attitudes as opposed to writing it in documents and policy. 
Because there are still many language teachers who are not very good at the language 
or intercultural issues and the question is how to educate these teachers, enhance their 
intercultural skills instead of putting it in the polices. Because, as I said, in the case of 
etwinning it is in the policy and this results that thousands of people register in etwinning 
but then they do nothing because they could not find the right project or right partner or 


because they don’t want [to]”. 


This idea of having a more ‘bottom-up’ approach to upscaling the use of virtual exchange was 


shared by another teacher in a separate interview: 


“We cannot impose top-down experiences; teachers need to know why and to believe in it 


in order to want to explore new practices; and they also need to be assured”. 


Finally, a third of the interviewees mentioned linking virtual exchange to the internationalisation 
programmes of universities and possibly integrating virtual exchange with physical mobility 


programmes: 


“Perhaps I can recommend telecollaboration as a component of future study abroad 


programs in the partner country?”. 


As regards recommendations for regional and national ministries, it was striking to see the 
general lack of awareness among teacher trainers as to how decision makers at these levels could 
actually contribute to or influence the promotion of virtual exchange in initial teacher education. 
Those who did respond to this question made suggestions related to including virtual exchange 
in the curriculum for primary and secondary school and that this would have a knock-on effect 
of making the activity more relevant in initial teacher education as well. Others suggested that 
the ministries should use their influence to shape the curriculum in initial teacher education 
institutions in order to give more emphasis to online learning in general. One practitioner made 


this suggestion: 


“They [the ministries] should definitely encourage institutions, universities to participate 
in virtual exchanges projects. When it comes to teacher education, it already says in the 
curriculum that ICT skills are vital but they should also encourage it more in teacher 
training... So in [my training institution] for example, there are no courses on technology 


which they [the student teachers] have to take”. 
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Teachers also mentioned the need for more training — both short-term (as in ‘get-started-quick’ 


workshops) and longer, continued education courses. 
5.7. Concluding comments 


The most outstanding feature of the interviews were the participating teachers’ affirmations 
that the exchanges had been beneficial to both their students and themselves, followed by 
straightforward declarations that designing and implementing such exchanges inevitably 
hold some challenges. However, all of the participants were equally adamant that none of the 
obstacles were insurmountable — in fact they should be embraced since many of their issues 
were simply outcomes of innovating their teaching practices. The interviewees underscored the 
need for bringing in and involving more teachers from their institutions as well as the necessity 
of fostering close collaboration with their students in the virtual exchanges and learning to 
mediate intercultural exchanges. Finally, the need for stronger institutional support — through 


both material and human resources — was highlighted. 
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The following case studies are intended to provide a more nuanced and detailed insight into 
how virtual exchange functions in initial teacher education and to gain an understanding of 
the factors which shaped the learning outcomes of each exchange. These are also intended to 
provide the reader who is unfamiliar with virtual exchange with some accessible examples of 


how the activity works in practice. 


The first of the case studies focusses on the intercultural learning outcomes of a Brazilian- 
Portuguese exchange which was carried out in Portuguese. The second, a German-Polish 
exchange, was carried out through English and the study focusses on the digital-pedagogical 
competence development of the students. Finally, the third exchange looks in detail at the foreign 
language learning experiences of Spanish and Swedish students who collaborated together. 
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SECTION 6. CASE STUDIES: VIRTUAL EXCHANGE IN ACTION IN INITIAL TEACHER EDUCATION 


6.1. A Brazilian-Portuguese exchange 
on the theme of inclusion in education 


6.1.1. Classes involved 


This case study involves a three-month long exchange between classes in Portugal and Brazil on 
the topic of inclusion in education. Both classes were at master level and students were studying 
special needs in education. The Portuguese class had nine students and the Brazilian class eight 
students. The Brazilian class also had a focus on sports education. Most of the students were 
also working part-time, which meant that they had limited time to dedicate to this project. The 
project received institutional recognition in Portugal, as it was approved by the Scientific Board 


of the Escola Superior de Educacdao. 


The project started late in the semester because the Portuguese teachers’ original partner 
dropped out of the project at the last minute but she was able to find a new class for the exchange, 
and despite starting late in the semester, and the fact that in Brazil holidays started from the 
middle of the exchange, the project was considered by the teacher trainers involved to have 


been extremely successful. 


The Portuguese teacher had no experience of virtual exchange but she had attended the 
EVALUATE workshop in Padova and collaborated in some of the mentoring meetings of other 
exchanges that other teachers at her university were involved in. Her institution was very 
supportive and had a small team of teachers working on virtual exchange. The Brazilian class 
had three teachers involved, and despite joining the project late, they managed to set up a 
good collaboration with their Portuguese partner. The teachers collaborated intensely during 
the exchange. They reported having weekly Skype meetings to discuss activities students were 
doing, and shared Google Docs and also said they were in touch almost daily on WhatsApp or 
Facebook. They also integrated the exchange into class activities and dedicated class time every 


week to discuss how the exchange was going — as their EVALUATE mentor had strongly advised. 


Despite the potential challenges that this partnership posed (i.e. its late start, the geographic 
distance and hence also time difference between the classes, and its timing during the holidays), 


through their intense collaboration, the teachers managed to make this project a success. 


The data from the diaries tell us that the majority of respondents reported coming from a 


homogeneous environment, as the following quote shows: “Very restricted. I had little contact 
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with people from other cultures, that’s why I found it difficult to reply to some questions 
due to my lack of knowledge concerning dialogue and exchanges with people from different 


cultures”. 


However, they wanted to make the most of the virtual exchange experience to acquire new 
knowledge and perspectives on education aiming to implement them in their own contexts. In 


response to the question regarding their expectations, students responded: 


“Know new technologies to use in the classroom, interact with other people and know 
about their cultures. Contribute to my professional training, as well as to that of the 


colleagues”. 


“T would like to learn other strands of education in order to have other working methods, 


as well as explore other ways of learning, bringing benefits to people with special needs”. 


“Understand and compare how another country works, its rules and culture. By making 
comparisons with Portugal to make conclusions about how we could improve our own 


country/work. Exchange ideas and learn new knowledge”. 
6.1.2. How the virtual exchange was run 


The Moodle platform was used to coordinate the exchange and for students to upload their 
work, but the students were also encouraged to use other forms of communication to interact 
with their peers during the exchange. Students were also required to do a collaborative writing 


task using a wiki. 


The exchange was on the specific theme of inclusion entitled Politicas Publicas de Inclusao em 
Portugal e no Brasil— semelhancas e diferencas (Public Policies of Inclusion in Portugal and Brazil 
— similarities and differences) and the teachers adapted the EVALUATE task sequence to promote 


student reflection about the specific content they were addressing. 


Considerable time was dedicated at the outset of the project on getting to know one another 
as the teachers felt it was important for the students to get to know one another well. Students 
made individual video or photo introductions for one another which they posted on Moodle, and 
they were required to comment on those of their international peers. They also had a class-to- 


class Skype video conference at the beginning of the exchange. 
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The theme of the exchange was inclusion of children with disabilities in public schools, and 
both countries have specific policies on this. These policies were explored in the second task. 
The groups analysed how inclusion in the two countries is structured, focussing on various 
comparable aspects. The groups had to search for texts, photos, facts, and interviews that 
illustrated and enriched their presentations. For their final task, they had to collaboratively 


prepare a presentation. 


Both classes also reflected on the telecollaborative experience, focussing on what they learned 
from the experience regarding the use of technology and virtual exchanges in the development 
of their professionalism, and also on their intercultural development. The exchange could thus 
be said to have implemented principles of ‘reflective, experiential learning’ (Kolb, 2015), with the 
guided reflections supporting the student learning during the process of the exchange. Teachers 
regularly used class time to discuss the exchange with the students, and also asked them to 


reflect on the process. 
6.1.3. Learning outcomes 


The statistical results were very positive in terms of change in technical knowledge and intercultural 
effectiveness, for which there was the highest gain of all the EVALUATE partnerships. This does 
not mean that they were the most interculturally effective on the scale, but that they had made 
the most progress in comparison with the other groups. In fact, the respondents’ scores on the 


intercultural effectiveness scale were below average both before and after the exchange. 


As reported earlier, the majority of students came from somewhat homogeneous backgrounds 
and had had few international and intercultural experiences. For them, unlike most of the 
other exchanges in the EVALUATE project which involved communication in a second language 
(usually English), these students were communicating in their first language, Portuguese. This 
may have lowered the barriers towards interacting with their peers and also allowed them to 
engage more deeply in the interactions about the issues they were studying, which require quite 


a high level of language competence to address in depth. 


The student responses to the journal entries and the teacher interviews confirmed the positive 
learning outcomes of the project in cognitive, attitudinal, and behavioural terms. The students 
were very interested in the theme which they learnt more about both in their own contexts as 
well as in their partners’. Hence, they perceived an increase in knowledge about the topic of 


diversity and inclusion: 
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“The investigation into accessibility allowed me to deepen my knowledge in its themes 
and in addition it is directly related to the wiki tool/online work. Overall, I valued the 


opportunity which I was given to reflect on the questions raised while working”. 


“We explored the thematic of diversity, when I realised that this topic covers endless issues: 
cultural, social, ethnical, linguistic, humane, religious, among others. One important 
point is the relationship between inclusion and diversity, because in schools, since the 
Convention on the Rights of Persons with Disabilities (2008) many children and adolescents 
entered school with the most diversified needs and is a teacher’s duty the management as 
a whole in order to work with these differences. Regarding the similarities between the 
two countries, we reached an agreement that in order to live in a plural society be it in 
regards to the ethnic, cultural, intellectual and/or physical differences, there is a need to 


respect different groups and cultures that there are within a country”. 


For the majority of the students, the main lesson learned was the possibility of doing collective 
writing and the online exchange of cultural experiences which for many was challenging, but 


also proved rewarding: 


“The collaborative text was the most remarkable experience due to its joint production, 


which included our opinions and experiences which complemented each other”. 
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“It was a pedagogical challenge which allowed for different ways of learning about 
different places (without being physically present in the place). Skype is an interesting 


tool for distance learning”. 


Use of class-to-class video conferencing, both at the beginning and at the end of the project, was 
also an important factor to community and relationship building, as reported by several of the 


students: 


“On the first assignment, we used images and a video in order to communicate with each 
other, in addition to Skype which brought the groups closer together. I’ve learnt that 
sometimes we restrict ourselves to the resources we use in our daily lives, as for example, 
I had never communicated via Skype before, nor introduced myself through video. The 
Moodle platform also allows for this type of approximation and communication, and is an 


easy and user-friendly tool”. 


At the end of the exchange, students reflected on what they have learned about the ways in 
which technology might influence their teaching approach and stressed the importance of using 


class video conferencing and tools that foster teamwork: 


“Regarding technology, I can see that the use of videos and video conferencing can capture 
people’s attention and foster dialogue, so I would use it in class. The presentation of videos 
as a strategy for discussion and dialogue with people who are studying or working on the 


same topic”. 


Though they did not give explicit examples, the students reported becoming aware of different 
perspectives on the same topic, and some saw this beyond purely cultural terms, but also 
highlighted the relevance of social and economic factors. In terms of intercultural awareness, 
this opening to different perspectives and recognition of diversity within their own societies 


marks an important step away from ethnocentrism: 


“T think it is very good because we could combine different conceptions about the same 
object. So it opened up spaces for different ways of exploring the same content, for 


instance”. 


“Itis good to see a topic being explored by a great variety of viewpoints. Different viewpoints 


consolidate the general opinion about the object of study”. 
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“The cultural barriers only exist if that would be our choice. In every country, knowledge 
is constructed through the same orientations, what differentiates are the social, economic 


and cultural particularities”. 


For one of the students it opened them up to the idea of going abroad, which they had not 


considered before: 


“It’s the first time that I do something with foreign partners and this contributed in a 
significative way to my personal and professional journey/career. Today I feel more 
comfortable in sharing my knowledge and also my own doubts with my Portuguese 
colleagues. In addition, I believe that this opens up the doors, as I start wondering about 
going to Portugal and have the opportunity to learn more with the colleagues/professors 


from there”. 


The Portuguese teacher also reported on the students’ and teachers’ desire to carry out a study 


visit after this exchange: 


“The exchange was carried out with a country (Brazil) where it is not possible to set up 
Erasmus exchange, but the Brazilian group is now with intention to try to carry out a ‘study 
visit’ to Castelo Branco next year. Teachers are also considering joint projects around this 


theme”. 


She also highlights the importance of virtual exchanges for students who are not mobile, and as 
mentioned in the introduction to this report, for a series of reasons, many students of education 


are not: 


“For these students, who are almost all from blue-collar families, it is not possible to go 
on an Erasmus exchange, so these virtual exchanges are of extreme importance because 
it affords them contact with other cultural realities that are very different from their 


country, which otherwise would be impossible”. 


From the teachers’ perspective, the EVALUATE project contributed to the exchange of knowledge 
concerning the thematic contents addressed, the technologies used, as well as the acquisition 
of intercultural competences as students had the opportunity to get to know new realities and 
cultures. They expressed a strong interest in continuing their experience of virtual exchanges 


despite the additional workload it placed on them. 
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6.2. A German-Polish exchange on the theme 
of digital-pedagogical competence 


6.2.1. Classes involved 


In this case study we focus on an exchange between 11 student teachers from a master course 
in a Teaching English to Speakers of Other Languages (TESOL) teacher training programme at a 
Polish university, and eight students of a similar master TESOL at a German university. The group 
consisted of 16 female and three male student teachers aged from 20 to 24. All student teachers 
had extensive pedagogical preparation, with most of them having done teaching practices. They 
were a highly disparate group in terms of intercultural communicative competence, language 
proficiency, and digital competence, which impacted their communication, task perception, and 


pedagogical knowledge in terms of task design. 


The virtual exchange took place during the 13-week winter term of the academic year 2017- 
2018. The participants worked in intercultural teams throughout the entire period. Weekly 1.5 
hour meetings in the local classrooms were spent as follows: (1) 45 minutes in local teams for 
reflection on the ongoing exchange and the learning experience and (2) 45 minutes spent online 


in the intercultural teams working on tasks. The participants used English as a lingua franca. 
6.2.2. | How the exchange was run 


The design of the virtual exchange followed a task sequence which is based on the progressive 
exchange model (O’Dowd & Ware, 2009) covering the three phases of information exchange, 
comparing, and analysing cultural practices, and working on a collaborative product. In their 
international groups, student teachers focussed on the collaborative design and peer evaluation 
of intercultural online-based and technology-supported tasks for their future English as a 
foreign language learners in primary and secondary schools. Following the experiential learning 
model (Hoven, 2006), student teachers experimented with various tools to get a feeling for their 


methodological use before they started the collaborative task design in Phases 2 and 3. 


During the information exchange, or getting-to-know-each-other phase, they posted a short 
multimodal introduction in a Padlet (virtual wall) so that all participants could be seen in one 
single space and to create a group feeling. Introductions were also posted in the moodle group 
forum. In their international groups, student teachers communicated synchronously via the 


video-conferencing tool Zoom, discovering its potential for direct oral negotiation while deciding 
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on a group name such as ‘The Detective Eyes’. In this particular case, all team members liked 
British crime novels and the British accent. For their written communication, the participants 
used Google Docs which allowed them to jointly draft a group philosophy based on their chosen 


name while discussing specifics in the comments which can be inserted into Google Docs. 


In the second task phase, the international teams analysed an online task facilitating 
intercultural learning, developed by student teachers in another virtual exchange, discussing 
the use of technology in the task, and making suggestions for improving task design and 
technology use. The analyses provided the backdrop for the teams’ own task designs for 
technology-mediated intercultural learning, thus fostering their competence development. 
Next, the international teams gave each other feedback on their respective tasks: both the 
design and the peer feedback was then presented in the local classrooms, thus allowing for 


comprehensive reflection on the task design process and its outcomes. 


This process was repeated in the third phase — technology-based task design — peer-feedback, 
local presentation, and reflection. However, this time, a more complex technology-based task 
sequence was designed accompanied by a website (using Weebly) to display the finished task 
sequence. Apart from using weebly, teams were required to integrate at least three digital 
tools into their task sequence. The intensive collaboration allowed student teachers to develop 


different aspects of TPACK to varying levels. 
6.2.3. Learning outcomes 


Student teachers’ positive results in terms of digital-pedagogical competence development as 
reflected in the quantitative data are corroborated by the qualitative data: diary entries, task 
products, and transcripts of joint reflection and classroom discussions that took place throughout 
the virtual exchange in both local contexts. Several themes emerged from the qualitative data. In 


what follows, we highlight instances that are indicative for competence development. 


It transpires from the student teachers’ diary entries that — before taking part in the virtual 
exchange - there was uncertainty, reluctance, and a lack of knowledge with regards to online 


tools and applications and their classroom integration: 


“I was not too optimistic about using online tools in class (like Weebly, Zoom, learning 
apps...), because I didn’t quite know how to incorporate those kinds of things in class. Now 


Iam sure that I am going to use them in class”. 
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“[...T]hanks to this project I realised how important is the use of technology in a teacher’s 
job. I found out information about many new tools and how to use them, in what purpose. 


I think this knowledge will be useful for me in the future”. 


Curricular expectations to work with online tools and applications in the classroom often 
generate high levels of anxiety among student teachers. It is therefore paramount to alleviate 
such anxieties by developing TPACK competences through virtual exchange. The fact that the 
participants reflected on these expectations at the very end of the virtual exchange confirms 
the need for such competence development in initial teacher education. In their final class 
discussion, student teachers brainstormed aspects they found helpful, interesting, challenging, 
or surprising in terms of digital-pedagogical, intercultural and foreign language competence 
developed during the virtual exchange. They then chose one critical incident which they had to 
draw (writing was not allowed) into the magnifying glass (see Figure 31). All magnifying glasses 
were put on the board in the local face-to-face classroom and student teachers could choose one 
drawing (not their own) and describe and interpret what they thought the drawing represented 
in terms of lived experiences during the virtual exchange. Finally the interpretations were 


discussed including feedback from the authors of the drawings. 
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student teacher) 


In the example below, a student teacher had brainstormed the following points under digital- 


pedagogical competence: 
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finding ways to incorporate technology in class; 
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through Zoom, Google Docs, Padlet etc.). 


While drawing her incident into the magnifying glass, she focussed on the challenge of 


incorporating technology in her future professional life as a classroom teacher. 


In the interpretation of this drawing, the group first thought it was a university classroom. 


However, the following exchange ensued: 


Instructor: Do the others agree, university context? 

Student 1: No, I don’t think so. Because so many students. I think it’s school, but maybe 
it’s about the technical skills. The teacher wants something from the students about the 
technical things and they just don’t know how to do it. 

Instructor: Who did this? You did this? Tell us. 

Student 2: This was a much better idea [pointing to student 1, laughter]. I don’t really 
know what I was thinking when I drew it, but I thought of all the preparation we had to 
do, especially with Weebly, with all the instructions we had to think of, not only the task 
instructions, but also how to use the technology, that this situation [points to the drawing] 
doesn’t happen. 

Instructor: Ah okay, so school classroom. 

Student 2: Yes, because we had to think about it when we did the Weebly, how our students 
would do it. So that is the picture. 

Instructor: So you see yourself here? 

Student 2: Totally. 

Instructor: As the future teacher, in front of your 20 eighth graders. 

Student 2: I was not so creative today... 

Instructor: No, that’s great. It’s important because you projected it into the future. Did you 
have the feeling with the Weebly that you covered the situation? 

Student 2: Yes. 

Instructor: So you feel confident about it? 


Student 2: Yes. 
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In sum: while carrying out the task of creating a Weebly page — which is a challenge for her - this 
student teacher considers what engaging her future students in a similar task might look like. 
However, having experienced the process during the virtual exchange, she now feels confident 


about dealing with such challenges in her future classroom. 


Apart from gaining confidence in using online tools in their future classrooms, Exchange 1 
participants also developed an understanding of the pedagogical potential of some of the tools, 


such as Padlet for example: 


“T especially liked Padlet. I think it is great for open discussions or the presentation of 


results”. 


Diary entries also underline the potential for authenticity which they associate with using 


technology in the classroom: 


“Tt offers the opportunity to stage real-life encounters inside the classroom where students 
are supposed to use the [foreign language] under real-life conditions in an authentic 


situation (when using Zoom, for example)”. 


Broadly speaking, Exchange 1 diary entries referenced as ‘Projected benefit of technology use 
(self)’ fall into two categories: projected benefit of using specific tools versus projected benefit 
of using technology in general. As for the latter, an increase in confidence and inspiration as to 


how technology can be integrated into the classroom stands out: 


“T have now more ideas how to apply online tools in the lesson”. 


At the same time, some student teachers mention that they have become more careful with 
regards to technology use, which is probably due to their heightened level of awareness of tool 


affordances: 


“T have been provided with many interesting online tools that will help me attract learners’ 
attention to the lesson and make the process of learning more pleasurable. Also, I have 


become more cautious about the choice of online technologies”. 


A closer look at Exchange 1 diary entries for ‘Projected benefit of technology use (students)’ 


reveals that comments related to the motivational benefit of technology use for students are 
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frequent. The potential for enhanced learner independence is also mentioned, as is the advantage 


of authentic interaction with other learners from around the globe: 


“If the teachers use e.g. controlled platforms, the students can do research all by themselves, 


individual or in teams, in order to do a task”. 


“Using technology in class offers a unique opportunity to have meaningful and especially 


authentic conversations with people from another part of the world”. 


Existing student familiarity with technology for communication and interaction in general and 
associated ease in terms of technology use for more formal educational purposes, also feature 
among student teachers’ reflections under this theme. However, we know from the literature 
that this is not necessarily the case. Already a decade ago, Selvyn (2009), for example, highlighted 
learners’ — especially young learners’ —- deficiencies in this respect: an inability to transfer digital 
skills acquired in informal contexts to formal ones. A lack of critical or evaluative skills among 
young learners (Sharpe, 2010) has also been brought to our attention, as well as the difficulties 
learners experience with taking a critical stance towards online content in learning contexts that 
include opinion-generating activities (Littlejohn, Beetham, & McGill, 2013) in particular, unless, 
of course, they have experienced relevant practices. As EVALUATE has demonstrated, virtual 
exchange embedded into the curriculum and implemented by those who have experienced it as 
part of their initial teacher education (‘experiential modeling’), offers relevant practices in this 


respect. 


Some reflections from Exchange 1 sit at the interface between ‘Projected benefit of technology 
use (students)? and ‘Methodological use of tools’, those that include both direct and indirect 


considerations of tool affordances in particular, e.g.: 


“the way how learners can present their work. Sometimes it’s easier and better for them 
to present short text or photo on the Padlet Wall, but when their ‘product’ is more complex 


it’s better to use presentation (e.g. Power Point)”. 


Student teachers developed a good feel for the potential or projected benefit of specific tools, 
also in terms of the interrelationship between tool use and intercultural sensitivity. For example, 
they reflected on critical incidents from an American-German virtual exchange of two 10th 
grade classrooms, to foster the transfer of competences developed in the context of EVALUATE to 


their future classrooms. In the 10th grade virtual exchange, the American students had tried to 
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explain Thanksgiving to their German partners. However, not only were there great differences 
among the individual explanations, some were also outright incorrect. Asked how they would 
deal with such diverse answers in a face-saving way, one of the EVALUATE student teachers 


suggested the following: 


“They could just post it [the American explanations] on the Padlet, so that those who wrote 
these statements, could see by themselves that it is different, and then they could start 


communicating about it. I think that’s better than if I just tell them that you are wrong”. 


Seeing their (anonymous) explanations on Padlet, the American partners will realise that they 
need to look at their explanations of Thanksgiving again, and negotiate cultural representation 


among themselves and then communicate this to their German partners. 


The student teacher was able to suggest this face-saving way of using Padlet because she had 
experienced a similar use of Padlet in her virtual exchange. Participants had been asked a few 
weeks into the exchange to anonymously post what bothered them most in the virtual exchange. 
They were also asked to make suggestions for solving problems posted by other student 
teachers, thus promoting an open exchange about contentious issues that had arisen: workload 


distribution, missing forum posts, perceived lack of interest, etc. 


In her reflections, this student teacher uses the competence she has developed as a forward 


organiser for her future school classroom. 
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Most references for ‘Methodological use of tools’ are instances where student teachers - after 
mentioning a bespoke tool or application used in their virtual exchange —- comment on the 
affordances of the tool or application and/or expand on their tool preference by providing a 


rationale: 


“Especially Padlet is a really good tool that I would like to use in the future as well: it makes 


it very easy to collect students’ opinions and to make them start a conversation”. 


“Using Google Docs is a great tool! I REALLY like it. It makes it so much easier for students 
to talk about a task and to work on it together. Whenever you cannot talk to each other via 
video chat or talk, Google Docs saves you. Everyone can write down his/her ideas at the 
same time. Thanks to the chat [commenting function] which you can use discussing about 
all the ideas straight away is really easy. If you use Google Docs, the times of writing a text, 
sending it to everyone else, waiting until they have read everything and worked on the 
text as well, are over! Exchanging ideas and working on tasks together even though you 


are not in the same place has never been that easy for me”. 


This also echoed in this student teacher’s classroom reflection as the following extract of the 


transcript shows: 


“When I tried it for the first time it was actually really confusing. But actually, this is a 


great tool to use whenever you have to cooperate and work on something together”. 


This observation is mirrored in student teachers’ exchanges in their virtual exchange groups 
discussing this tool when evaluating another virtual exchange team’s use of technology in 
their task design (task phases 2 and 3). The evaluating team becomes aware of the missed 
opportunity for communication among international partners, pointing out that Padlet 
may also afford exchanges about task products, thus allowing for further commenting and 


negotiation. 


Student 1: At the end of Phase 2 there is an idea that they put their suggestions on the 
Padlet wall and that’s ok. But then, at the beginning of Phase 3 students are asked to present 
their findings, and I think that it’s not necessary because they can just read their partners’ 
suggestions on Padlet. Maybe they should read it and comment instead of repeating their 
thoughts. 


Student 2: I agree, it’s unnecessary 
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Student 3: Yes I agree! 

Student 2: Perfect! 

Student 3: Oh yes you are right! We could praise the idea of the Padlet as a new tool...) 
Student 1: Yeah 


In another instance of providing feedback, a team suggests improvements of using specific tools 


in the classroom, pointing out their affordances and challenges: 


“Storybird is a cool method, but you can’t include your ‘own’ pictures, just the ones that 


are provided on Storybird and it takes a while to understand the platform”. 


“Good to introduce them to new options like Storybird, just takes time to create one. Maybe 


PowerPoint or Piktochart would be easier and they could add photos”. 


Methodological use of tools and finding out how to prepare learners for informed tool use is a 
crucial aspect of digital-pedagogical competence. In the following example, the student teacher 
provides very positive feedback on a complex task sequence designed by another virtual 
exchange team. She points out that the task sequence is well designed in terms of preparing 


students for what she perceives to be a challenging session in Zoom: 


“T really liked your lesson. I found that there was a really sequencing task and one task build 
up on the other. And I felt like that if I was a student, I could imagine that they can actually 
follow the procedure very well. You started to work with questions in the beginning and 
you used these questions later on in the while and post-task. I thought that students will be 
well prepared at the point when they get to Zoom with their partners. And they won’t have 
a problem to actually communicate with their partners. Because they have ideas in mind 
about questions. And I found that very important because it would be bad, I think when they 
Zoom and they didn’t know what to talk about and what to ask about”. 


She also highlights the importance of sufficient support for students expected to use a new tool, 
by sharing a link to an online tutorial, for example, or by having the teacher model its use for 
the students. In this way she shows her appreciation for a pedagogical informed approach when 


dealing with technology. 


“What I liked a lot, was your support for the teacher, for the About me page. First of all, 


you gave them the link of the About me page,... And also, the teacher presents his or her 
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own About me page. So, I liked this because it can serve as a model for the students. And 
you, as far as I remember, you said that students will get personal support when they 


create their own About me. I found that good”. 


All these examples speak to the student teachers’ development of technological pedagogical 
knowledge through virtual exchange. Overall, the comments from this exchange speak to the 
fact that systematic training combining technology and pedagogy such as the virtual exchanges 


offered in the context of the EVALUATE project, remains a desideratum. 


6.3. ASpanish-Swedish exchange on the theme 
of primary school education 


6.3.1. Classes involved 


This is a case study of an exchange between Spanish and Swedish students. Both groups were 
studying to become primary school teachers, which means that the EVALUATE task they completed 
was Primary connections: a telecollaborative exchange for future primary school teachers. 
Quantitative data places the group of learners studied here in 13th position (out of 25) for the 
learning gains they reported making in digital-pedagogical competence, in 15th position (again 
out of 25) for their learning gains in intercultural communicative competence, and in seventh 
position for their learning gains in foreign language competence. Averaged across the three 
competences, it is clear that these scores make them a representative, rather than an outstanding 
group, with an above average performance in terms of foreign language development. This makes 
them ideal for this particular case study, where predominant focus will be on what participants 
reported having learned in relation to foreign language competence. The data sources, which 
offer a basis for the case study, are five post-exchange interviews conducted with small groups of 
between two to four students of Spanish participants, plus two post-exchange oral presentations 
delivered by similar groups of Swedish participants, in which they reflected specifically on what 


they felt they had learned (or not) in the course of the exchange. 


The classes involved in this exchange were large. The Spanish group consisted of 44 second 
year students; the Swedish group was 25 strong. The teachers who organised the exchange had 
varying degrees of expertise. The teacher of the Spanish group was a seasoned telecollaborator 
with more than two decades of experience of organising virtual exchanges. His Swedish 
counterpart was relatively new to virtual exchange. In the circumstances, they performed 


remarkably well. 
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6.3.2. | How the exchange was run 


Both groups participated in the exchange as part of an existing classroom-based programme. 
In the case of the Spanish participants, this was an obligatory module in English as a Foreign 
Language and Teaching Methodology. The course as a whole was worth six European Credit 
Transfer and Accumulation System (ECTS) credits, while participation in the exchange was 
worth 2.5 ECTS credits. Some Spanish participants expressed disquiet that the amount of 
credits awarded to their Swedish counterparts was higher than that which they received. In 
fact, for the latter, the exchange was worth 7.5 ECTS, but the award of these was conditional 
on their also passing the examinations in the course. When interviewed, the Spanish students 
made a telling plea for parity of treatment for both groups of participants: “[w]e could say that 
it is good if we all have to do the same. I mean... it has the same importance for everyone”. 
Ideally, one assumes, this would apply not only to credit rating, but also to synchronising - as 
far as possible - the imparting of information and instructions to both groups. This speaks 
to a particular dilemma for organisers and facilitators of virtual exchanges. The diversity of 
institutional structures is inevitable and discovering and understanding other institutional 
microcultures is arguably a key outcome of virtual exchanges. But the fact that they exist and 
the incomprehension to which they can give rise is a challenge that requires awareness and 


negotiation. 


As indicated above, the overall task engaged in by participants was designed for primary school 
teachers. In common with the other two tasks, this comprised three successive types of activity, 
involving: (1) information exchange, (2) comparing and analysing cultural practices, and (3) a 


collaborative product (e.g. a joint lesson plan). 


Though there is a strong pedagogic rationale for this succession of activity types, both Spanish 
and Swedish students found it challenging to fit all three into the six-week period they had for 
the exchange. In fact ‘time’ was the topic which recurred most frequently in both interviews and 
presentations. A Spanish interviewee urged organisers of future exchanges “to give students 
more time to do the tasks, and maybe those that are different, that are more difficult, like, for 
example, the last task- more time”. The suggestion made by her Swedish counterpart is different, 


but motivated by the same sense that participants were pressed for time: 


“We think that the tasks should maybe be shorter and more focussed, because there isa time 
aspect of working with this telecollaboration that shouldn’t be underestimated, because 


connecting to other schools, it’s time-consuming. And the projects, if they’re shorter and 


THE EVALUATE GROUP 1 0 3 


more focussed, there’d be a greater possibility of actually finishing something... Maybe 
if we were given Task 3 in the beginning, we feel, and just forget about learning to know 


each other? We can do that as we go along”. 


The technologies used by students included a dedicated Moodle virtual learning environment, 
Google Docs, and WhatsApp. Interviewed after the exchange, two Spanish participants described 
WhatsApp as “a useful app to keep in touch”. In particular, they identified “the immediacy 
that it offers” as a major benefit and compared this favourably to the Moodle virtual learning 
environment, “because people forget to enter the Moodle”. Their approval of WhatsApp did not, 
however, prevent them from showing a critical awareness of its possible drawbacks as a learning 
environment, namely the inability to ensure a balanced level of contribution from participants: 
“[t]he disadvantage in WhatsApp is like, sometimes people write too much and if you have a group, 
it can be annoying. But the advantage is that you reach the people really fast”. Another participant 
expressed unease on the grounds that the context in which she normally used WhatsApp was 
personal and informal: “[w]hen you’re with your phone you use WhatsApp to entertain, to talk to 


my friends. Not like a thing to learn. It’s like a thing I cannot match with education”. 


While WhatsApp use may not therefore have been entirely unproblematic, what emerged most 
clearly from participants’ comments on technologies was the limited usefulness of a standard 
learning management tool, such as a Moodle virtual learning environment, in a situation where 


time was of the essence. A Spanish participant made this point with particular clarity: 


“In my personal opinion, Moodle is not the best way to carry on all the work, because we 
never know when somebody is uploading something, or is making a comment. We don’t 
know when to reply. So it’s better to have a WhatsApp group, or at least Google Docs, to 
work, everybody together, so that we can have comments and they get sent to our emails, 


so we know how the work is progressing”. 

6.3.3. Learning outcomes 
The learning outcomes listed by participants are varied. They fall into three broad areas: 
organisational and time management skills, open-mindedness and empathy, and, finally, 


politeness and precision in language use. 


Virtually all interviewees mentioned the need to be organised and to manage one’s time. 


When asked what she had learned, one Spanish participant responded simply “we learned 
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that organisation is really important”. Another classmate concurred, affirming that “[t]o 
have organisation is essential. And to start the task as soon as possible”. These are, of course, 


transferable skills, the possession of which makes a graduate highly employable. 


In terms of personal attributes and attitudes, participants stressed that open-mindedness was 
both a requisite for successful participation in virtual exchanges as well as a likely outcome 
of that participation. The former view was expressed with admirable maturity by a Spanish 


participant: 


“If somebody has an idea and another person from another country has the complete 
opposite idea, they have to give in to each other a little bit..., so that they can work together 
for the project. Otherwise, if you’re fixed in your idea and not open-minded, there’s nothing 


to do, because you won’t be satisfied with your work either way”. 


That view was echoed by her Swedish counterpart, who saw her and her classmates’ own 
increased openness to innovative, technologically-based pedagogy as a key gain from their 


participation in an online exchange: 


“After taking part in this telecollaboration project, we feel that we have learned more 
about technology that can be gained out of students across the world and we believe that 


we would most likely be more open to using telecollaboration in future”. 


Other participants went even further than advocating mere openness and stressed the need to 


be able to put oneself in a partner’s shoes: 


“T would say, try to put yourself into the other person’s shoes. Try to understand where 
they are coming from. I think that’s important also, because sometimes we tend to focus 
too much on our culture and what we know and we forget that they come from a different 
culture, they have maybe different values from us. And that’s not negative or positive, it’s 


something you have to keep in mind, I think”. 


Effectively, what is being advocated here is a capacity for empathy, which is both a personal 
attribute and an interpersonal and intercultural competence. The hallmark of any fully-rounded 
human being, it is also an indispensable employability skill for the global workplace, where 


multicultural teamwork is now the norm. 
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One of the features of this exchange was that it was exclusively textual. This brings particular 
challenges, since text on its own is a relatively restricted medium, lacking all the non-verbal cues 
which ease oral-aural communication, even when mediated by videoconferencing. The difficulty 
of communication purely by text was clearly recognised by participants. A Spanish interviewee 


noted: “I think that all the interactions were written, which was not very good”. 


Accordingly, the exchange was the scene of at least one critical incident, which occurred in a 
discussion of LGBTQ questions. A Spanish participant asked if it was possible for members of the 
LGBTQ community to display affection publicly for each other in public places, on the grounds 
that, in some parts of Spanish cities, this might expose gay couples to homophobic attacks. His 
question was clumsily expressed, in that he wrote of LGBTQ people “hiding who they really 
are”. This evoked an extremely hostile response from two Swedish interlocutors, whereupon he 
apologised and offered an explanation of his question. In reply, a Swedish participant expressed 
the view that “this is an example o[f] an intercultural discussion” and expressed a readiness to 
engage in further “intercultural communication exchange”. To this observer, it is not clear that 
there was a significant difference in the views on sexual orientation held by the two groups. 
The incident appears much more likely to have been an example of miscommunication, rather 
than cultural dissonance. What it illustrates, however, is how fundamental the role played by 


language (and by the need for linguistic precision) is in virtual exchanges. 


The need for politeness and precision was something that participants themselves clearly 
understood. Spanish participants stressed both the importance of providing feedback to one’s 
partners and the need to ensure that, in doing so, one did not give offence: “I had to take special 
care when I said things. I looked to- to [sic] be sure that I was being polite. and that [no] one could 


get offended by me”. 


Swedish partners in this exchange were equally aware of the requirement to adjust their use of 
the foreign language to the communicative context. As one of them put it: “[t]he students from 
Spain were not maybe at the same level as us, I don’t know, but that’s a part of it, adjusting our 
language to making them understand, so there can be a dialogue, and I think we had a really 


great dialogue with them”. 


If greater pragmatic control was one aspect of foreign language competence development 
undergone by participants, so too were attempts to ensure the greater linguistic precision which 


contributes to such control. Here, there is abundant evidence that Spanish participants were just 
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as linguistically aware as their Swedish partners. As one of them expressed it: “[y]ou have to 
revise what you’re going to send, because we’re very used to oral communication and it’s not 
the same. And also try to be as polite as possible”. This was far from an individual insight. That 
is clear from the responses of some of her fellow interviewees. Asked what advice they would 
give to those taking part in similar exchanges, their responses are telling. One urges: “[p]lease, 
guys, be careful with the choice of communication” while her co-interviewee responds “[y]eah, 
be careful when you write... think twice”. There could be no better advice for anyone planning 


to communicate online. 
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7. Outcomes and conclusion 
of the EVALUATE study 


7.1. Conclusions and implications of the study 


This study brought together the experiences of more than a thousand students of initial teacher 
education and 51 teachers who took part in 25 projects based on the same model of virtual 
exchange. The different parts of this study looked at the impact of these virtual exchanges on three 
competence sets — intercultural, digital-pedagogical, and foreign language — and also examined the 
experiences of the teacher trainers who integrated these exchanges into their classes; but what 
overall conclusions can be extrapolated from these different studies? Several themes emerged 
repeatedly from our findings which allow us to draw some general conclusions on the potential of 


virtual exchange for initial teacher education and for other areas of university education. 


7.1.1. Virtual exchange is a popular and effective educational tool which acts 


as a driver for new learning opportunities for both learners and teachers 


Perhaps the most important finding which can be drawn from this European policy experiment 
is that class to class virtual exchange is a potent tool for promoting innovation and international 
learning in the university classroom. The majority of both students and teachers confirmed 
that they had found virtual exchange to be an enriching and enjoyable learning activity in 
their courses and that they would recommend extending its use to other courses. Our study 
also demonstrated that engaging students in structured online intercultural collaboration as 
part of their formal learning contributes to the development of students’ digital-pedagogical, 
intercultural and foreign language competence sets. This was seen in the quantitative studies 
which showed steady growth in both intercultural and TPACK (digital-pedagogical) scores in 


both iterations of our study as well as in the qualitative studies. 


However, the qualitative analysis of students’ learner diaries and the interviews with the 
teachers involved in the study also revealed that virtual exchange is at its most effective 
when it forces learners and teachers out of their comfort zones and brings them to engage in 
linguistic, intercultural, and technological learning experiences which they would not usually 
be confronted with in their day-to-day learning. The qualitative study of intercultural learning 
outcomes, for example, revealed that virtual exchange best enhanced students’ intercultural 


competence because the participants were confronted with a range of collaborative hurdles 
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and challenges which required them to find creative ways to collaborate and communicate 
successfully with their international partners. The impact of virtual exchange was seen to be 
particularly significant on students who came from homogeneous backgrounds and who were 
not accustomed to interacting and collaborating with members of other cultures. For these 
students it was important to be exposed to, and begin to engage with, perspectives different 
from their own. Virtual exchange was an important step in raising their curiosity and openness 
towards others, and has given many of them the confidence and sparked their interest in seeking 


more opportunities to engage with difference and further develop their intercultural sensitivity. 


The qualitative study of digital-pedagogical competence also reflected the value of virtual 
exchange in pushing students to experiment with new communication technologies and to 
then reflect critically on how these technologies can be used in innovative ways in educational 
contexts. The study on foreign language development demonstrated how virtual exchange 
contributes to students’ ability to interact and understand others. Significantly, students also 
reported large gains related to confidence in using the foreign language. As in the case of 
intercultural learning, this appeared to be due to the fact that, for many undergraduate students 
coming from homogenous backgrounds, this was the first experience they had of using a foreign 
language (in this case, English) to communicate with speakers of other languages. While foreign 


language education may be a common element in most initial teacher education programmes 
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across Europe, students have little opportunity to use their foreign languages in any form of 
authentic context unless they engage in periods of physical mobility. Virtual exchange can offer 
a low-cost, classroom-based alternative for those who cannot engage in physical mobility or as 


preparation for those who plan to do so in the future. 


Most teachers who implemented the exchanges also reported being very satisfied with the 
experience and that the experiences had been positive for both themselves and their students. 
Similarly to their students, teachers found that virtual exchange had forced them to go further in 
their teaching and had obliged them to be more innovative in their classrooms, introducing more 
participative teaching strategies. It had also led them to explore other international learning 


opportunities, including new physical mobility agreements and student visits. 


7.1.2. Virtual exchange is a complex learning activity which requires integration 


into a formal educational framework as well as guidance from educators 


While virtual exchange may be both popular and effective, evidence emerged from all datasets that 
engaging students in online intercultural collaboration projects is a challenge for both students 
and teachers, and that the activity required integration into students’ classes and being considered 
in their course evaluation procedures. The intercultural study highlighted the importance of 
providing students with support so that they will engage with difference on more than merely a 
superficial level, while the study on digital-pedagogical competence underlined the importance 
not only of engaging students in online communication using different digital tools, but also of 
giving them the opportunity to reflect, under the guidance of their teachers, on the affordances 
of these tools and how they could be applied in their own teaching contexts in the future. To deal 
adequately with these challenges, the teachers interviewed in our study recognised that integrating 
the exchanges into their classes and providing some form of academic recognition for students’ 
work were vital. They also pointed out that organising and running such a complex activity also 
requires a great deal of time and work on their own behalf and that institutions should find ways 


to recognise this work in the same way that other academic activity is recognised. 


7.1.3. Virtual exchange may be mediated by digital technologies 


but its success depends on person-to-person engagement 


This is perhaps an obvious statement but the importance of establishing fluid and respectful 
working relationships between partner teachers and between students and their international 


partners should not be taken for granted by educators and programme designers as they establish 
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virtual exchanges in their institutions. While educators may tend to focus on the logistics of the 
exchanges and the pedagogical tasks which students will work on together, our data across the 
different studies has served as a reminder of the importance of providing opportunities for 
teachers and students to establish good working relationships together, to get to know their 
partners as real people, and not merely as anonymous names and avatars on a computer screen. 


This has to be attended to in virtual exchange programme design. 


In our data, students regularly pointed out that the task-based focus of the exchanges and 
the text-based nature of the communication had led to a feeling that their collaborations 
were depersonalised and that they lacked the feeling of interacting with ‘real people’. They 
often overcame this barrier by using communication tools which they used regularly in their 
everyday lives such as WhatsApp and Messenger to communicate with their partners, while 
teachers also reported that regular videoconferencing had helped students to establish good 
working relationships together. Thanks to videoconferencing, students reported being able to 
get a better sense of who their partners were. Videoconferencing was also considered key by 
students when it came to successful language learning. In exchanges where students did not 
have an opportunity to interact with their partners in live video interactions, their evaluation of 


their foreign language development was often quite low. 


Another important step to helping develop good working relationships was seen to be the 
integration of activities which allowed students to get to know each other on a social level. The 
use of ‘getting to know you’ activities encouraged students to keep in mind that their partners 


were also people with their own needs, pressures, hopes, and aspirations. 


What was the case for the students was also seen to be true for the teachers as they strove to 
develop good working relationships with their partner teachers. Successful teacher partnerships 
were seen to stem from having had the opportunity to get to know each other in a face-to-face 
context or via videoconference before the exchange and, second, by maintaining regular contact 


via email or videoconferencing with the partner teacher during the exchange itself. 


7.1.4. The growth of virtual exchange depends on the provision of increased 


training and support from institutions and educational decision makers 


Our European policy experiment has demonstrated that virtual exchange is a powerful learning 


tool which gives students and teachers the opportunity to access innovative international 
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learning opportunities, but we have also seen that it is a challenging and complex activity which 
requires educators to develop an array of organisational, pedagogical, and intercultural skill 
sets in order to implement these exchanges successfully. For this reason, our final conclusion is 
that institutions of initial teacher education and educational authorities at regional, national, 
and European levels must be willing to support the process through the provision, first of all, 
of funding to train teachers and to enable them to travel to meet and plan with their partner 
teachers; and second, through ensuring the availability of the necessary time and technological 


infrastructure to develop and implement their online international projects. 


Virtual exchange can prove to be an important part of internationalisation at home and a potent 
complement to physical mobility, but it requires that teachers are trained, that their additional 
workload is recognised and that their institutions have the necessary institutional, technological, 


and administrative support necessary to carry out exchanges. 


7.2. The impact of EVALUATE at regional, 
nation, and European level 


One of the key characteristics of the Erasmus+ KA3 European policy experiments is the attempt 
to bring together both researchers and policy makers in order to maximise the potential impact 
and scalability of innovative policy measures. In EVALUATE, the participating ministries of 
education from Castilla y Le6n, Baden-Wtrttemberg, Hungary, Spain, and Portugal worked 
closely with the research team to set the research objectives and then to explore how virtual 
exchange could best be promoted in teacher education across their regions and countries and 


across Europe. 


In terms of establishing objectives, the team of researchers and ministerial representatives 
identified the shared concern of the participating regions and countries that the increased 
availability of online technologies in teacher education was not being accompanied by the 
integration of student-centred, collaborative approaches to teaching. In our initial meetings, the 
team established that all the participating ministries have policy documentation and initiatives 
which highlighted this concern. These included Baden-Wirttemberg’s policy document 
E-Learning (Strategische Handlungsfelder der Hochschulen des Landes Baden-Wiirttemberg zur 
Digitalisierung in der Hochschullehre) and the Spanish Ministry’s document (Marco Comun de 
Competencia Digital Docente) as well as the Portuguese Technological Plan for Education. All of 


these coincided in the urgency to address the following needs: 
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future teachers need to be trained to use online technologies to implement 


learner-centred, collaborative pedagogies; 


universities and other initial teacher education institutions need to put into 
practice more innovative training methods to give future teachers first-hand 


experience of online collaborative learning; and 


greater sharing of good practices between countries is necessary in the 


training and certification of teachers in innovative teaching methodology 


involving ICTs. 


During the project, the ministries provided not only reports which served to guide the 
research team in how the research should be structured, but also interviews which enabled 
the researchers to establish the impact of the study. In these interviews, it became clear that 
the ministries identified virtual exchange as an ideal tool for developing various key areas in 
higher education. For example, one member of the German public authority explained “[t]he 
issues dealt with by EVALUATE are very important as they fit in our strategy of digitalisation 
in Baden-Wurttemberg. This project is important because [virtual exchange] combines several 
competencies which are vital in teacher training such as internationalisation, intercultural 
competencies and so on”. Another member from Portugal explained: “[s]tudent teachers don’t 
only need digital competences, they need to know how to develop them in their own students. 


So this type of learning needs to be included in initial teacher education”. 


The public authorities often highlighted the complexities of their education systems and the 
limitations which they faced when trying to influence and shape the methodological approaches 
used in initial teacher education in their countries and regions. The member of the Ministry 
of Science, Research and the Arts in Baden-Wutrttemberg pointed out that “in Germany in 
general, the universities are autonomous organisations so we can only promote examples of 
good practice”, while a colleague from the Portuguese Ministry of Education explained “[m] 
y ministry doesn’t have responsibility over higher education institutions [...] but we are ina 
position to disseminate the results of this research and to encourage that telecollaboration can 
be integrated into initial teacher education”. Similarly, the Ministry of Human Capacities in 
Hungary also reported that while it was not within their ability to introduce virtual exchange 
in any direct way, they were in a position to promote and recommend its use in Hungarian 
institutions of initial teacher education. They did this by promoting the regional workshop on 


virtual exchange organised in Hungary and also by contacting the national representatives of 
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eTwinning in Hungary to make them aware of the work of this project. The Hungarian ministry 
also reported that they would be explicitly recommending their institutions of initial teacher 
education to become familiar with the methodology of virtual exchange and to apply it in their 


programmes. 


Nevertheless, developments in the final months of the project have shown they were able 
to use the findings of the European policy experiment to develop their ministerial strategies 
and to inform their initiatives related to initial teacher education. For example, the Ministry 


113 


1 1 4 SECTION 7. OUTCOMES AND CONCLUSION OF THE EVALUATE STUDY 


of Education of Castilla y Leén reported that it has established a working group to develop a 
dialogue between the regional educational administration and the colleges and faculties of 
education of the universities in this autonomous community. The purpose of this working group 
was to design proposals related to the innovation and the improvement of the educational system 
of Castilla y Leon, especially as regards teaching competences and initial teacher training. The 
findings of the EVALUATE project were presented to this working group in its first meeting and 
they will be incorporated into the recommendations the group will publish in their final official 
document which will be presented to the Spanish Ministry of Education. The aim here will be to 
promote the use of virtual exchange in primary and secondary schools of the region as well as 
promoting the use of virtual exchange in initial teacher education in Spain as well as in other 


regions and countries. 


EVALUATE was also seen by the participating ministries as an excellent approach to promoting 
collaboration between public authorities and universities. For example, the Minister of 
Education for Castilla y Le6n made reference to the project in a speech at the University of Leén 


on 7 September 2018: 


“T consider this University as an example of good practices in Education. The Faculty 
of Education shows a great potential with a young and passionate staff that is bringing 
innovation to the classrooms. The teaching training programmes are focussed on 
digitalisation, multilingualism and inclusiveness, three cornerstones of our policy in the 
Ministry. I must mention here the close collaboration of one of my departments, the General 
Directorate of Universities and Research with the European research project, EVALUATE, 
led by the University of Ledn. The symbiosis between research and policy making will be 
an important issue to be developed in the drafting of a document that will be drawn next 
year by a working group of the Ministry of Education, and that will be sent to the Central 
Ministry of Education in Madrid to be taken into account in the next State Agreement on 


Education”. 


7.3. How can we support 
virtual exchange and extend 
its use in initial teacher education? 


Based on the findings of our study, we would make the following recommendations to the 


authorities and decision makers working in the area of virtual exchange. 
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Higher education institutions can... 


increase the number of teacher-training programmes on the theme of virtual 
exchange for staff and encourage staff participation in initiatives such as 


Erasmus+ Virtual Exchange; 


offer teachers extra credit or time release for carrying out virtual exchange in 


their classes; 


make an explicit mention of participation in virtual exchange in students’ 


European Diploma Supplement; 


award students credits for participating in virtual exchange; and 


make virtual exchange part of their internationalisation at home strategy. 
Regional and national ministerial authorities can... 


provide funding for further teacher training programmes for teacher 


educators in their countries/regions; 

incorporate virtual exchange into the public authorities’ projects and 
initiatives; 
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and strategy papers; and 


promote virtual exchange to counterparts in other regions/countries and to 


other educational decision makers in the European Union. 


The European Union and Erasmus? can... 


raise awareness of virtual exchange as a valid part of universities’ 
internationalisation policies and ensure its integration into the Erasmus 


Charter; 


provide funding for virtual exchange ‘planning visits’ for teachers to meet and 


work with their partner teachers; 


offer specific funding for projects and research initiatives related to the 


activity; and 


encourage greater collaboration between virtual exchange initiatives at 


university level and initiatives such as Erasmus+ Virtual Exchange. 
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8. Concluding comments 


It was seen at the outset of this report that European policy documents on initial teacher 
education underline the need for 21st century teachers not only to be digitally literate but also 
to be able to engage their students in online learning activities which are based on collaborative 
and intercultural principles of learning. The European education monitor, for example, reports 
that currently in European schools “ICT is mostly used as a remedial tool” and “few teachers 
report using ICT for communication in which students are involved, leaving untapped the 


potential to connect students together” (European Commission, 2015b, p. 59). 


In response to this situation, this study presented virtual exchange, an approach to online 
learning which simultaneously engages learners in digital and international learning and which 
is clearly based on the principles of collaborative learning. The European policy experiment 
carried out the largest study of class-to-class virtual exchange to date which involved 25 virtual 
exchange field trials bringing together institutions of initial teacher education in 16 countries. 
The study found that not only did students and teachers find virtual exchange to be a highly 
positive learning experience, but they also believed the experience would be useful for their 
future careers as teachers. The study also established that virtual exchange contributes to the 
development of students’ digital, intercultural, and foreign language competence sets. While the 
quantitative learning gains of the study were quite modest, the qualitative findings allowed us to 
demonstrate how this method opens up students to new international experiences, helps them to 
learn by overcoming communicative and organisational barriers and challenges, and gives them 
first-hand experience of using digital tools for educational purposes. Many of the participants 
in the study reported coming from quite homogeneous societies and virtual exchange offered 
them their first experience of working in an international team. Participants reported building 
confidence and losing personal fears through the exchange, learning to negotiate with peers who 
have different opinions or ways of thinking, managing to see issues from different perspectives, 


and solving problems. 


The teacher trainers also confirmed to us that participating in a virtual exchange provided 
their students with invaluable intercultural learning experiences as well as the opportunity 
for authentic use of their foreign languages. The teachers reported that the virtual exchange 
had impacted on their own professional practices by providing them with opportunities to 
collaborate with colleagues internationally, to innovate their classes and to improve their own 


teaching methods. 
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However, the study and its accompanying case studies also clearly show that virtual exchange is 
not easy to implement and that it requires teachers to be aware of the principles of good practice 
which have been outlined in the pages of this report. These include, for example, ensuring 
the integration and academic recognition of virtual exchange in their study programmes and 
overcoming the barriers which distance and technology can put up in relationship building 


through the use of videoconferencing. 


Thanks to the training initiatives offered by EVALUATE and other European-sponsored projects 
such as EVOLVE and Erasmus+ Virtual Exchange, virtual exchange will undoubtedly continue 
to grow as a method in initial teacher education across Europe at a grassroots level; but for it 
to have large-scale impact and sustainability, the role of public authorities and the European 
Commission are vital. Decision makers need to refer to virtual exchange explicitly in their 
policy documents, to provide specific sources of funding for virtual exchange initiatives and 
to encourage universities to integrate virtual exchange into their internationalisation policies. 
Virtual exchange should be seen as a compliment to, not as competition or as alternative for, 
physical mobility initiatives. It was seen in this study that the public authorities that took part in 
EVALUATE have already taken steps to promote this method in their own regions and countries 
but much more remains to be done. We hope that this report will contribute to the achievement 


of this goal. 
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Virtual exchange refers to education programmes in which constructive communication and interaction 
takes place between individuals or groups from different cultural backgrounds with the support of educators 
or facilitators. Evaluating and  Upscaling Telecollaborative Teacher Education (EVALUATE, 
http://www.evaluateproject.eu/) was a European policy experimentation financed by Erasmus+ which 
studied the impact of a telecollaborative model of virtual exchange on student teachers. 


Between 2017-2018, the project consortium trained teacher trainers and organised virtual exchanges which 
involved over 1,000 student teachers at initial teacher education institutions. This entailed students 
interacting and collaborating with partner classes from other countries as an integral part of one of their 
courses. The research team then analysed the learning gains from these exchanges using qualitative and 
quantitative research methodologies. They also worked with representatives from European ministries of 
education to understand how virtual exchange could be upscaled in teacher education across Europe. 


This publication presents the findings of the EVALUATE experimentation and its implications for the 
education of future teachers. The study found that engaging student teachers in structured online 
intercultural collaboration as part of their formal learning can contribute to the development of their 
digital-pedagogical, intercultural, and foreign language competences. It can also lead to innovation and 
international learning in the education of future teachers. 
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